[jira] [Comment Edited] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444739#comment-13444739
 ] 

Uwe Schindler edited comment on LUCENE-4332 at 8/30/12 5:58 PM:


bq. Will this require people to put rhino in ant's lib/ folder? If so, I'd just 
use an ugly property value="-D..=${} -D..=${}..". Yours is very nice but an 
additional step is required which seems an overkill?

We already have a lot of script tasks in our build files (e.g. generating 
website using ). Javascript is shipped with every known JVM. - And pitest is 
only run by few people, much more call tasks like "root/lucene$> ant 
documentation" or "root$> ant svn-check-working-copy", so this is no issue here.

  was (Author: thetaphi):
bq. Will this require people to put rhino in ant's lib/ folder? If so, I'd 
just use an ugly property value="-D..=${} -D..=${}..". Yours is very nice but 
an additional step is required which seems an overkill?

We already have a lot of script tasks in our build files (e.g. generating 
website using ). Javascript is shipped with every known JVM.
  
> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3768) add some prelim assertions to OverseerTest

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated SOLR-3768:
--

Attachment: capture-1.jpg

A screenshot with a history of a single test from Bamboo.

> add some prelim assertions to OverseerTest
> --
>
> Key: SOLR-3768
> URL: https://issues.apache.org/jira/browse/SOLR-3768
> Project: Solr
>  Issue Type: Test
>Reporter: Hoss Man
> Attachments: capture-1.jpg, SOLR-3768.patch
>
>
> There isn't much i understand about OverseerTest, but today while doing a 
> full test run I got an unreproducible assertion failure from this line...
> {noformat}
> assertNotNull("could not find counter for shard:" + ids[i], ai);
> {noformat}
> ...in which the assertion message indicated that not only was "ai" null, but 
> "ids[i]" was null as well.
> Poking arround the test a bit, i think what's happening here is that some of 
> the preliminary logic in testShardAssignmentBigger has bounded wait loops to 
> "make sure " things have happened, but there is no assertion that these 
> things actually happen if that the loop bound is exhausted - which can lead 
> to missleading/confusing errors further on in the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3768) add some prelim assertions to OverseerTest

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444741#comment-13444741
 ] 

Dawid Weiss commented on SOLR-3768:
---

Thanks Hoss. I still have in mind that idea of yours to have a "failure 
history" for each test. Atlassian's Bamboo CI server has this built-in, don't 
know about jenkins -- perhaps there is a plugin for this somewhere? It'd be 
nice to have such a feature.

> add some prelim assertions to OverseerTest
> --
>
> Key: SOLR-3768
> URL: https://issues.apache.org/jira/browse/SOLR-3768
> Project: Solr
>  Issue Type: Test
>Reporter: Hoss Man
> Attachments: capture-1.jpg, SOLR-3768.patch
>
>
> There isn't much i understand about OverseerTest, but today while doing a 
> full test run I got an unreproducible assertion failure from this line...
> {noformat}
> assertNotNull("could not find counter for shard:" + ids[i], ai);
> {noformat}
> ...in which the assertion message indicated that not only was "ai" null, but 
> "ids[i]" was null as well.
> Poking arround the test a bit, i think what's happening here is that some of 
> the preliminary logic in testShardAssignmentBigger has bounded wait loops to 
> "make sure " things have happened, but there is no assertion that these 
> things actually happen if that the loop bound is exhausted - which can lead 
> to missleading/confusing errors further on in the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444743#comment-13444743
 ] 

Dawid Weiss commented on LUCENE-4332:
-

bq. Javascript is shipped with every known JVM

Umm, really? I didn't know about it, I thought a separate download is needed. I 
know the scripting API is part of JRE but is an implementation also embedded?

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444745#comment-13444745
 ] 

Dawid Weiss commented on LUCENE-4332:
-

Indeed, just checked, sorry for misdirecting, Greg, I didn't know js impl. is 
part of the default platform.

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Moved] (SOLR-3769) unit test failing in solr4 beta

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss moved LUCENE-4344 to SOLR-3769:
---

Lucene Fields:   (was: New)
  Key: SOLR-3769  (was: LUCENE-4344)
  Project: Solr  (was: Lucene - Core)

> unit test failing in solr4 beta
> ---
>
> Key: SOLR-3769
> URL: https://issues.apache.org/jira/browse/SOLR-3769
> Project: Solr
>  Issue Type: Bug
>Reporter: Gary Yngve
>
> ant test  -Dtestcase=LeaderElectionTest -Dtests.seed=3F5F939070D8F356 
> -Dtests.slow=true -Dtests.locale=sl_SI -Dtests.timezone=Africa/El_Aaiun 
> -Dtests.file.encoding=US-ASCII
> [junit4:junit4] ERROR   0.00s | LeaderElectionTest (suite)
> [junit4:junit4]> Throwable #1: java.lang.AssertionError: ERROR: 
> SolrZkClient opens=33 closes=28
> [junit4:junit4]>  at 
> __randomizedtesting.SeedInfo.seed([3F5F939070D8F356]:0)
> [junit4:junit4]>  at org.junit.Assert.fail(Assert.java:93)
> [junit4:junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:230)
> [junit4:junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:83)
> [junit4:junit4]>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
> [junit4:junit4]>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit4:junit4]>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4:junit4]>  at java.lang.reflect.Method.invoke(Method.java:601)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:754)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4]>  at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4]>  at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
> [junit4:junit4]>  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> [junit4:junit4]>
> [junit4:junit4] Completed in 221.77s, 4 tests, 1 failure, 1 error <<< 
> FAILURES!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444751#comment-13444751
 ] 

Uwe Schindler commented on LUCENE-4332:
---

Sun/Oracle, IBM J9 and jRockit contain it. Only the FreeBSD one is missing it 
because of Licensing reasons, but thats a well-known bug to be fixed soon (so 
on ASF Jenkins its installed as separate package in JRE_PATH/lib/ext" or like 
that).

The major drawback of the sjhipped javascript version is that it is trimmed 
down and produces no bytecode out of Javascript, it interprets only. But that 
is no issue here.

The Java 6 specs are a little bit unclear, but they say something like 
(unfortunately you have to download as ZIP file): A "default" scripting 
implementation is required to be shipped with any Java 6 JDK, but it is not 
defined which one. Oracle officially says "javascript (Rhino) is shipped with 
OpenJDK and JDK", so all other JVM vendors would be stupid to ship with another 
default one. So, let's assume its there, otherwise you have a problem building 
the release package of Lucene. It is similar to the case that JDK contains 
implementations for XML processing, but the language spec only enforce the API 
to be there and SPI to load them. Which XML processor is shipped depends on 
developer, but there must be one.

Another (bloated) version of doing that is using ivy:cachepath to download and 
add a classpathref= to the  element, but then you will at some time 
hit the stupid ANT permgen issue (if you have many  elements).

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444757#comment-13444757
 ] 

Uwe Schindler commented on LUCENE-4332:
---

junit4:pickseed is cool. I did not know that this is available to ant! Thanks 
Dawid!

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Greg Bowyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated LUCENE-4332:


Attachment: 
LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch

I found and fixed a few things relating to security policies (ha!)

Also I thought about the javascript a bit, I know thats its standard in the JVM 
but when I tested on a 1.6 I found that my javascript was a bit too advanced 
for it.

That made me pause for thought and realise that I might have been a bit too 
clever for my own good there; I am not all that happy with the the JS so I went 
with the ugly approach for now.

It think the right solution is to work with the pitest people and get it to 
support junit style nested tags like sysproperty

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4337) Create Java security manager for forcible asserting behaviours in testing

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444762#comment-13444762
 ] 

Uwe Schindler commented on LUCENE-4337:
---

Greg,
I have a good idea, that prevents us from really specifying everything in the 
policy file: I would only make the AllOtherPermission available in 
test-framework and make it take a list of java class names (comma separated) as 
parameter. Then it can be added to the default policy file like:

{noformat}
permission org.apache.lucene.utils.AllOtherPermission 
"java.io.FilePermission,java.net.SocketPermission,java.security.SecurityPermission";
{noformat}

This would grant (like yours) all other permissions, so only the listed 
permissions are added to the policy file. The implies() would only check for 
all those classes.

I will come up with a new patch, that simpliefies this!

> Create Java security manager for forcible asserting behaviours in testing
> -
>
> Key: LUCENE-4337
> URL: https://issues.apache.org/jira/browse/LUCENE-4337
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 4.1, 5.0, 4.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
> Attachments: ChrootSecurityManager.java, 
> ChrootSecurityManagerTest.java, LUCENE-4337.patch, LUCENE-4337.patch, 
> LUCENE-4337.patch
>
>
> Following on from conversations about mutation testing, there is an interest 
> in building a Java security manager that is able to assert / guarantee 
> certain behaviours 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444763#comment-13444763
 ] 

Dawid Weiss commented on LUCENE-4332:
-

bq. junit4:pickseed is cool. I did not know that this is available to ant! 
Thanks Dawid!

No problem, it was needed to assure reproducibility because we pick 
file.encoding randomly and it should use the same seed. Had I known about the 
fact javascript is embedded in every JDK I'd probably use that instead :)

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-3700) Create a Classification component

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili reassigned SOLR-3700:
-

Assignee: Tommaso Teofili

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3700) Create a Classification component

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444827#comment-13444827
 ] 

Tommaso Teofili commented on SOLR-3700:
---

the suggested snippet for calculating the frequency of terms (from the 
'content' field) in docs with a certain class is ok apart that the terms should 
be extracted from the text field instead of the class field and the docFreq 
should be counted on the class field:
{code}
Terms terms = MultiFields.getTerms(atomicReader, textFieldName);
long numPostings = terms.getSumDocFreq(); // number of term/doc pairs
double avgNumberOfUniqueTerms = numPostings / (double) terms.getDocCount(); 
// avg # of unique terms per doc
int docsWithC = atomicReader.docFreq(classFieldName, new BytesRef(c));
return avgNumberOfUniqueTerms * docsWithC; // avg # of unique terms in text 
field per doc * # docs with c
{code}
comparing the previous (slow) ranked search giving an output of 92 this gives 
an estimated output of ~98.6 which seems reasonable.

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3700) Create a Classification component

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444837#comment-13444837
 ] 

Shai Erera commented on SOLR-3700:
--

Is there any reason not to develop it as a Lucene module? I haven't looked at 
the patch, but if it's not Solr-specific, or depends on Solr API, perhaps we 
can make this issue a LUCENE- one?

I see no reason such module will be available for Solr users only, unless you 
plan to depend on Solr API, in which case I will not slow down your development 
by insisting it becomes a Lucene module.

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3700) Create a Classification component

[Chris Male (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444841#comment-13444841
 ] 

Chris Male commented on SOLR-3700:
--

bq. Is there any reason not to develop it as a Lucene module? I haven't looked 
at the patch, but if it's not Solr-specific, or depends on Solr API, perhaps we 
can make this issue a LUCENE- one?

+1

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3700) Create a Classification component

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444843#comment-13444843
 ] 

Tommaso Teofili commented on SOLR-3700:
---

The patch is Solr specific just because it uses a Solr RequestHandler to expose 
the Classifier interface, also the idea is that more classifier implementations 
(e.g. based on MLT) may be plugged in so I thought Solr was a good place for 
it, however it's ok for me to put this into a Lucene module and then add only 
the needed Solr specific bindings to use it in Solr.

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3770) Overseer can lose updates to cluster state

[Sami Siren (JIRA)]

Sami Siren created SOLR-3770:


 Summary: Overseer can lose updates to cluster state
 Key: SOLR-3770
 URL: https://issues.apache.org/jira/browse/SOLR-3770
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Sami Siren
Assignee: Sami Siren
 Fix For: 4.0


It seems there's a bug in overseer where it removes a work item from the 
primary queue before inserting it to the work queue. This can cause messages to 
be lost if overseer is killed in between removing and adding the message. 


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3770) Overseer can lose updates to cluster state

[Sami Siren (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sami Siren updated SOLR-3770:
-

Attachment: SOLR-3770.patch

Trivial patch that should fix the problem.

> Overseer can lose updates to cluster state
> --
>
> Key: SOLR-3770
> URL: https://issues.apache.org/jira/browse/SOLR-3770
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Sami Siren
>Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3770.patch
>
>
> It seems there's a bug in overseer where it removes a work item from the 
> primary queue before inserting it to the work queue. This can cause messages 
> to be lost if overseer is killed in between removing and adding the message. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3700) Create a Classification component

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444854#comment-13444854
 ] 

Shai Erera commented on SOLR-3700:
--

bq. however it's ok for me to put this into a Lucene module and then add only 
the needed Solr specific bindings to use it in Solr

if it doesn't complicate matters for you, then it will be great if you can do 
that !

> Create a Classification component
> -
>
> Key: SOLR-3700
> URL: https://issues.apache.org/jira/browse/SOLR-3700
> Project: Solr
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4339) Allow deletions on Lucene3x codecs again

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444884#comment-13444884
 ] 

Michael McCandless commented on LUCENE-4339:


+1, the patch looks good!

The delGen/delCount are separately tracked in SegmentInfoPerCommit,
the first time we read a 3.x index we read the legacy SegmentInfos
format into RAM with the proper delCount/delGen
(Lucene3xSegmentInfosReader.readLegacyInfos).  If any new deletes
happen against that segment, SIPC will be updated, and on commit we'll
write delGen/delCount just like we would for a non-3x index.  We use a
separate method (SegmentInfos.write3xInfo) to write just the
SegmentInfo for 3.x segments.

SIPC.files() will correctly show the new del gen filename on write
since we just use Lucene40LiveDocsFormat.



> Allow deletions on Lucene3x codecs again
> 
>
> Key: LUCENE-4339
> URL: https://issues.apache.org/jira/browse/LUCENE-4339
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Priority: Blocker
> Fix For: 4.0
>
> Attachments: LUCENE-4339.patch
>
>
> On dev@lao Hoss reported that a user in Solr was not able to update or delete 
> documents in his 3.x index with Solr 4:
> {quote}
> On the solr-user list, Dirk Högemann recently mentioned a problem he was 
> seeing when he tried upgrading his existing solr setup from 3.x to 4.0-BETA.  
> Specifically this exception getting logged...
> http://find.searchhub.org/document/cdb30099bfea30c6
> auto commit error...:java.lang.UnsupportedOperationException: this codec can 
> only be used for reading
>  at 
> org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.writeLiveDocs(Lucene3xCodec.java:74)
>  at 
> org.apache.lucene.index.ReadersAndLiveDocs.writeLiveDocs(ReadersAndLiveDocs.java:278)
>  at 
> org.apache.lucene.index.IndexWriter$ReaderPool.release(IndexWriter.java:435)
>  at 
> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:278)
>  at 
> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2928)
>  at 
> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2919)
>  at 
> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2666)
>  at 
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2793)
>  at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2773)
>  at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:531)
>  at org.apache.solr.update.CommitTracker.run(CommitTracker.java:214)
> Dirk was able to work arround this by completely re-indexing, but it seemed 
> strange to me that this would happen.
> My understanding is that even though an IndexUpgrader tool was now available, 
> it wasn't going to be required for users to use it when upgrading from 3.x to 
> 4.x.  Explicitly upgrading the index format might be a good idea, and might 
> make hte index more performant, but as I understood it, the way things had 
> been implemented with codecs explicitly upgrading the index format wasn't 
> strictly neccessary, and that users should be able to upgrade their lucene 
> apps same way that was supported with other index format upgrades in the 
> past: the old index can be read, and as changes are made new segments will be 
> re-written in the new format.  (Note in
> particular: at the moment we don't mention IndexUpgrader in MIGRATE.txt at 
> all.)
> It appears however, based on this stack trace and some other experiements i 
> tried, that any attempts to "delete" documents in a segment that is using the 
> Lucene3xCodec will fail.
> This seems like a really scary time bomb sitaution, because if you upgrade, 
> things will seem to be working -- you can even add documents, and depending 
> on the order that you do things, some "old" segments may get merged and use 
> the new format, so *some* deletes of "old" documents (in those 
> merged/upgraded) segments may work, but then somewhere down the road, you may 
> try to a delete that affects docs in a still un-merge/upgraded segment, and 
> that delete will fail -- 5 minutes later, if another merge has happened, 
> attempting to do the exact same delete may succeed.
> All of which begs the question: is this a known/intended limitation of the 
> Lucene3xCodec, or an oversight in the Lucene3xCodec?
> if it's expected, then it seems like we should definitely spell out this 
> limitation in MIGRATE.txt and advocate either full rebuilds, or the use of 
> IndexUpgrader for anyone who's indexes are non-static.
> On the Solr side of things, i think we should even want to consider 
> automaticly runnin

[jira] [Commented] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444887#comment-13444887
 ] 

Michael McCandless commented on LUCENE-4343:


+1

We had a lot of tokenizers abusing setReader for stuff they should be doing in 
reset!

It would be really nice to make setReader final but that sounds like a 
challenge...

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4339) Allow deletions on Lucene3x codecs again

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4339:
---

Attachment: LUCENE-4339.patch

I improved the test case a bit, and that uncovered a bug in how we upgrade a 
3.x SegmentInfo: we were not recording the marker file in SI's fileSet, which 
caused IFD to delete it and for the upgrade to keep happening every time commit 
is called.  I fixed it by adding the marker file up front before we write the 
.si.

> Allow deletions on Lucene3x codecs again
> 
>
> Key: LUCENE-4339
> URL: https://issues.apache.org/jira/browse/LUCENE-4339
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Priority: Blocker
> Fix For: 4.0
>
> Attachments: LUCENE-4339.patch, LUCENE-4339.patch
>
>
> On dev@lao Hoss reported that a user in Solr was not able to update or delete 
> documents in his 3.x index with Solr 4:
> {quote}
> On the solr-user list, Dirk Högemann recently mentioned a problem he was 
> seeing when he tried upgrading his existing solr setup from 3.x to 4.0-BETA.  
> Specifically this exception getting logged...
> http://find.searchhub.org/document/cdb30099bfea30c6
> auto commit error...:java.lang.UnsupportedOperationException: this codec can 
> only be used for reading
>  at 
> org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.writeLiveDocs(Lucene3xCodec.java:74)
>  at 
> org.apache.lucene.index.ReadersAndLiveDocs.writeLiveDocs(ReadersAndLiveDocs.java:278)
>  at 
> org.apache.lucene.index.IndexWriter$ReaderPool.release(IndexWriter.java:435)
>  at 
> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:278)
>  at 
> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2928)
>  at 
> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2919)
>  at 
> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2666)
>  at 
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2793)
>  at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2773)
>  at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:531)
>  at org.apache.solr.update.CommitTracker.run(CommitTracker.java:214)
> Dirk was able to work arround this by completely re-indexing, but it seemed 
> strange to me that this would happen.
> My understanding is that even though an IndexUpgrader tool was now available, 
> it wasn't going to be required for users to use it when upgrading from 3.x to 
> 4.x.  Explicitly upgrading the index format might be a good idea, and might 
> make hte index more performant, but as I understood it, the way things had 
> been implemented with codecs explicitly upgrading the index format wasn't 
> strictly neccessary, and that users should be able to upgrade their lucene 
> apps same way that was supported with other index format upgrades in the 
> past: the old index can be read, and as changes are made new segments will be 
> re-written in the new format.  (Note in
> particular: at the moment we don't mention IndexUpgrader in MIGRATE.txt at 
> all.)
> It appears however, based on this stack trace and some other experiements i 
> tried, that any attempts to "delete" documents in a segment that is using the 
> Lucene3xCodec will fail.
> This seems like a really scary time bomb sitaution, because if you upgrade, 
> things will seem to be working -- you can even add documents, and depending 
> on the order that you do things, some "old" segments may get merged and use 
> the new format, so *some* deletes of "old" documents (in those 
> merged/upgraded) segments may work, but then somewhere down the road, you may 
> try to a delete that affects docs in a still un-merge/upgraded segment, and 
> that delete will fail -- 5 minutes later, if another merge has happened, 
> attempting to do the exact same delete may succeed.
> All of which begs the question: is this a known/intended limitation of the 
> Lucene3xCodec, or an oversight in the Lucene3xCodec?
> if it's expected, then it seems like we should definitely spell out this 
> limitation in MIGRATE.txt and advocate either full rebuilds, or the use of 
> IndexUpgrader for anyone who's indexes are non-static.
> On the Solr side of things, i think we should even want to consider 
> automaticly running IndexUpgrader on startup if we detect that the 
> Lucene3xCodec is in use to simplify things -- we can't even suggest running 
> "optimize" as a quick/easy way to force and index format upgrade because if 
> the 3x index as already optimized then it's a no-op and the index stays in 
> the 3x format.

[jira] [Commented] (LUCENE-4339) Allow deletions on Lucene3x codecs again

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444892#comment-13444892
 ] 

Uwe Schindler commented on LUCENE-4339:
---

Cool, thanks. Was the bug also there before this patch (means other code 
merging old segments could be affected)?

> Allow deletions on Lucene3x codecs again
> 
>
> Key: LUCENE-4339
> URL: https://issues.apache.org/jira/browse/LUCENE-4339
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Priority: Blocker
> Fix For: 4.0
>
> Attachments: LUCENE-4339.patch, LUCENE-4339.patch
>
>
> On dev@lao Hoss reported that a user in Solr was not able to update or delete 
> documents in his 3.x index with Solr 4:
> {quote}
> On the solr-user list, Dirk Högemann recently mentioned a problem he was 
> seeing when he tried upgrading his existing solr setup from 3.x to 4.0-BETA.  
> Specifically this exception getting logged...
> http://find.searchhub.org/document/cdb30099bfea30c6
> auto commit error...:java.lang.UnsupportedOperationException: this codec can 
> only be used for reading
>  at 
> org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.writeLiveDocs(Lucene3xCodec.java:74)
>  at 
> org.apache.lucene.index.ReadersAndLiveDocs.writeLiveDocs(ReadersAndLiveDocs.java:278)
>  at 
> org.apache.lucene.index.IndexWriter$ReaderPool.release(IndexWriter.java:435)
>  at 
> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:278)
>  at 
> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2928)
>  at 
> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2919)
>  at 
> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2666)
>  at 
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2793)
>  at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2773)
>  at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:531)
>  at org.apache.solr.update.CommitTracker.run(CommitTracker.java:214)
> Dirk was able to work arround this by completely re-indexing, but it seemed 
> strange to me that this would happen.
> My understanding is that even though an IndexUpgrader tool was now available, 
> it wasn't going to be required for users to use it when upgrading from 3.x to 
> 4.x.  Explicitly upgrading the index format might be a good idea, and might 
> make hte index more performant, but as I understood it, the way things had 
> been implemented with codecs explicitly upgrading the index format wasn't 
> strictly neccessary, and that users should be able to upgrade their lucene 
> apps same way that was supported with other index format upgrades in the 
> past: the old index can be read, and as changes are made new segments will be 
> re-written in the new format.  (Note in
> particular: at the moment we don't mention IndexUpgrader in MIGRATE.txt at 
> all.)
> It appears however, based on this stack trace and some other experiements i 
> tried, that any attempts to "delete" documents in a segment that is using the 
> Lucene3xCodec will fail.
> This seems like a really scary time bomb sitaution, because if you upgrade, 
> things will seem to be working -- you can even add documents, and depending 
> on the order that you do things, some "old" segments may get merged and use 
> the new format, so *some* deletes of "old" documents (in those 
> merged/upgraded) segments may work, but then somewhere down the road, you may 
> try to a delete that affects docs in a still un-merge/upgraded segment, and 
> that delete will fail -- 5 minutes later, if another merge has happened, 
> attempting to do the exact same delete may succeed.
> All of which begs the question: is this a known/intended limitation of the 
> Lucene3xCodec, or an oversight in the Lucene3xCodec?
> if it's expected, then it seems like we should definitely spell out this 
> limitation in MIGRATE.txt and advocate either full rebuilds, or the use of 
> IndexUpgrader for anyone who's indexes are non-static.
> On the Solr side of things, i think we should even want to consider 
> automaticly running IndexUpgrader on startup if we detect that the 
> Lucene3xCodec is in use to simplify things -- we can't even suggest running 
> "optimize" as a quick/easy way to force and index format upgrade because if 
> the 3x index as already optimized then it's a no-op and the index stays in 
> the 3x format.
> {quote}
> Robert said, that this is a wanted limitation (in fact its explicitely added 
> to the code, without that UOE it "simply works"), but I disagree here and 
> lots of other people:
> {

[jira] [Resolved] (SOLR-3770) Overseer can lose updates to cluster state

[Sami Siren (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sami Siren resolved SOLR-3770.
--

Resolution: Fixed

I committed this, trunk: r1378892, 4x: r1378895
 

> Overseer can lose updates to cluster state
> --
>
> Key: SOLR-3770
> URL: https://issues.apache.org/jira/browse/SOLR-3770
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Sami Siren
>Assignee: Sami Siren
> Fix For: 4.0
>
> Attachments: SOLR-3770.patch
>
>
> It seems there's a bug in overseer where it removes a work item from the 
> primary queue before inserting it to the work queue. This can cause messages 
> to be lost if overseer is killed in between removing and adding the message. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3731) Disallow null CoreContainer in ZkController constructor

[Sami Siren (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444898#comment-13444898
 ] 

Sami Siren commented on SOLR-3731:
--

I changed the wait time to match what is currently in ZkController and the 
OverseerTest passes now (after fixing SOLR-3770 too) on my superslow vm.

> Disallow null CoreContainer in ZkController constructor
> ---
>
> Key: SOLR-3731
> URL: https://issues.apache.org/jira/browse/SOLR-3731
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Sami Siren
>Assignee: Sami Siren
>Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-3731.patch
>
>
> Currently some tests use null CoreContainer when using ZkController directly 
> in tests and ZKController has to deal with this. I'd like to remove this 
> possibility.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444906#comment-13444906
 ] 

Robert Muir commented on LUCENE-4343:
-

I think we can still make it final: we just have 2 or 3 bad apples to figure 
out :)


> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Chris Male (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444908#comment-13444908
 ] 

Chris Male commented on LUCENE-4343:


+1 to the improvements and pursuing making it final.

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4337) Create Java security manager for forcible asserting behaviours in testing

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444922#comment-13444922
 ] 

Uwe Schindler commented on LUCENE-4337:
---

I tried that out but failed to do it, because the policy file was not able to 
load the custom Permission at all (it must be signed maybe). It did not even 
print a System.err inside a static {} block in the class!

I also think the current explicit permissions are much better to manage and we 
know what our code needs (as we have to enable additional permissions when new 
features are added).

> Create Java security manager for forcible asserting behaviours in testing
> -
>
> Key: LUCENE-4337
> URL: https://issues.apache.org/jira/browse/LUCENE-4337
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 4.1, 5.0, 4.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
> Attachments: ChrootSecurityManager.java, 
> ChrootSecurityManagerTest.java, LUCENE-4337.patch, LUCENE-4337.patch, 
> LUCENE-4337.patch
>
>
> Following on from conversations about mutation testing, there is an interest 
> in building a Java security manager that is able to assert / guarantee 
> certain behaviours 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4337) Create Java security manager for forcible asserting behaviours in testing

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444924#comment-13444924
 ] 

Uwe Schindler commented on LUCENE-4337:
---

I will commit the attached patch now to trunk and 4.x.

> Create Java security manager for forcible asserting behaviours in testing
> -
>
> Key: LUCENE-4337
> URL: https://issues.apache.org/jira/browse/LUCENE-4337
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 4.1, 5.0, 4.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
> Attachments: ChrootSecurityManager.java, 
> ChrootSecurityManagerTest.java, LUCENE-4337.patch, LUCENE-4337.patch, 
> LUCENE-4337.patch
>
>
> Following on from conversations about mutation testing, there is an interest 
> in building a Java security manager that is able to assert / guarantee 
> certain behaviours 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4337) Create Java security manager for forcible asserting behaviours in testing

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-4337.
---

Resolution: Fixed
  Assignee: Uwe Schindler  (was: Greg Bowyer)

Committed trunk revision: 1378914
Committed 4.x revision: 1378915

Thanks Greg for the good idea and some help + hints :-)

> Create Java security manager for forcible asserting behaviours in testing
> -
>
> Key: LUCENE-4337
> URL: https://issues.apache.org/jira/browse/LUCENE-4337
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 4.0-BETA
>Reporter: Greg Bowyer
>Assignee: Uwe Schindler
> Attachments: ChrootSecurityManager.java, 
> ChrootSecurityManagerTest.java, LUCENE-4337.patch, LUCENE-4337.patch, 
> LUCENE-4337.patch
>
>
> Following on from conversations about mutation testing, there is an interest 
> in building a Java security manager that is able to assert / guarantee 
> certain behaviours 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4337) Create Java security manager for forcible asserting behaviours in testing

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4337:
--

Affects Version/s: (was: 4.0)
   (was: 5.0)
   (was: 4.1)
   4.0-BETA
Fix Version/s: 4.0
   5.0

> Create Java security manager for forcible asserting behaviours in testing
> -
>
> Key: LUCENE-4337
> URL: https://issues.apache.org/jira/browse/LUCENE-4337
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 4.0-BETA
>Reporter: Greg Bowyer
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: ChrootSecurityManager.java, 
> ChrootSecurityManagerTest.java, LUCENE-4337.patch, LUCENE-4337.patch, 
> LUCENE-4337.patch
>
>
> Following on from conversations about mutation testing, there is an interest 
> in building a Java security manager that is able to assert / guarantee 
> certain behaviours 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

Nagaraj Molala created SOLR-3771:


 Summary: While using RSS indexing from Solr, we are getting error 
"Caused by: java.net.UnknownHostException" & indexing fail.
 Key: SOLR-3771
 URL: https://issues.apache.org/jira/browse/SOLR-3771
 Project: Solr
  Issue Type: Bug
  Components: search
Affects Versions: 3.1
 Environment: Solr Search engine using RSS data feeding
Reporter: Nagaraj Molala


we are getting below error. Please give us the solution as this is a show 
stopper for our application.
 
https://issues.apache.org/jira/browse/SOLR 2:51 PM 
Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at sun.net.NetworkClient.doConnect(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
   at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
   at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
   at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
e.java:97)
   ... 13 more

 


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b51) - Build # 750 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/750/
Java: 32bit/jdk1.8.0-ea-b51 -client -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 19165 lines...]
-jenkins-javadocs-lint:

[...truncated 24 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:252: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:110: The 
following files are missing svn:eol-style (or binary svn:mime-type):
* lucene/tools/junit4/tests.policy

Total time: 26 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b51 -client -XX:+UseG1GC
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b51) - Build # 750 - Failure!

[Uwe Schindler]

Falling in my own trap... :-)

I fixed this. It was caused by my development (I reverted changes, tried 
something else and reapplied the original patch...). So I committed the missing 
eol-style.

Uwe
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: Policeman Jenkins Server [mailto:jenk...@sd-datasolutions.de]
> Sent: Thursday, August 30, 2012 3:35 PM
> To: dev@lucene.apache.org; u...@thetaphi.de; markrmil...@apache.org
> Subject: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b51) - Build # 
> 750 -
> Failure!
> 
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/750/
> Java: 32bit/jdk1.8.0-ea-b51 -client -XX:+UseG1GC
> 
> All tests passed
> 
> Build Log:
> [...truncated 19165 lines...]
> -jenkins-javadocs-lint:
> 
> [...truncated 24 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:252: The following
> error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:110: The
> following files are missing svn:eol-style (or binary svn:mime-type):
> * lucene/tools/junit4/tests.policy
> 
> Total time: 26 minutes 57 seconds
> Build step 'Invoke Ant' marked build as failure Recording test results
> Description set: Java: 32bit/jdk1.8.0-ea-b51 -client -XX:+UseG1GC Email was
> triggered for: Failure Sending email for trigger: Failure
> 



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3699) SolrIndexWriter constructor leaks Directory if Exception creating IndexWriterConfig

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444959#comment-13444959
 ] 

Mark Miller commented on SOLR-3699:
---

hmm...I have to look closer than just at the patch, but the setDirFactory 
method seems a little troubling - why do we do that instead of making it part 
of the constructor? What if you don't set it?

{code}
+if (null != directoryFactory) {
+  // :HACK: normally we rely on updateHandler to do this, 
+  // but what if updateHandler failed to init?
+  directoryFactory.close();
+}
{code}

I don't think you want that? The dir factory should and will be closed by the 
DefaultSolrCoreState when it's ref count hits 0 - you don't actually want to 
close it when closing a core.

Otherwise, the basic idea seems fine - if the IW fails, release it's dir.

> SolrIndexWriter constructor leaks Directory if Exception creating 
> IndexWriterConfig
> ---
>
> Key: SOLR-3699
> URL: https://issues.apache.org/jira/browse/SOLR-3699
> Project: Solr
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-3699.patch, SOLR-3699.patch, SOLR-3699.patch, 
> SOLR-3699.patch
>
>
> in LUCENE-4278 i had to add a hack to force SimpleFSDir for 
> CoreContainerCoreInitFailuresTest, because it doesnt close its Directory on 
> certain errors.
> This might indicate a problem that leaks happen if certain errors happen 
> (e.g. not handled in finally)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.6.0_34) - Build # 531 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/531/
Java: 32bit/jdk1.6.0_34 -server -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 18319 lines...]
-jenkins-javadocs-lint:

[...truncated 24 lines...]
BUILD FAILED
C:\Jenkins\workspace\Lucene-Solr-trunk-Windows\build.xml:252: The following 
error occurred while executing this line:
C:\Jenkins\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:110: The 
following files are missing svn:eol-style (or binary svn:mime-type):
* lucene/tools/junit4/tests.policy

Total time: 46 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Description set: Java: 32bit/jdk1.6.0_34 -server -XX:+UseConcMarkSweepGC
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Jack Krupansky (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444978#comment-13444978
 ] 

Jack Krupansky commented on SOLR-3771:
--

Is there any specific reason why you believe that this is a bug in Solr? I 
mean, "unknown host" could simply be a DNS or firewall issue that you have to 
resolve at your own site - how have you verified that this is not the case?


> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.1
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b51) - Build # 750 - Failure!

[Robert Muir]

On Thu, Aug 30, 2012 at 9:42 AM, Uwe Schindler  wrote:
> Falling in my own trap... :-)
>
> I fixed this. It was caused by my development (I reverted changes, tried 
> something else and reapplied the original patch...). So I committed the 
> missing eol-style.
>

Generics Policeman vs. Robocop

-- 
lucidworks.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3731) Disallow null CoreContainer in ZkController constructor

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444990#comment-13444990
 ] 

Mark Miller commented on SOLR-3731:
---

So far so good for me.

> Disallow null CoreContainer in ZkController constructor
> ---
>
> Key: SOLR-3731
> URL: https://issues.apache.org/jira/browse/SOLR-3731
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Sami Siren
>Assignee: Sami Siren
>Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-3731.patch
>
>
> Currently some tests use null CoreContainer when using ZkController directly 
> in tests and ZKController has to deal with this. I'd like to remove this 
> possibility.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Invalid XML Output (Arabic):

[Steve McKay]

Are you sure this isn't an Eclipse issue? The plaintext (quoted-printable) 
source of the  elements looks like this:

0

AFAIK this is valid XML. There are only a few codepoints not allowed by XML 
1.0. I don't see a right-to-left mark, so it seems like Mail.app is trying to 
infer a right-to-left mark for the Arabic text and getting it wrong, causing 
the mangled display.

On Aug 29, 2012, at 2:17 PM, Fuad Efendi  wrote:

> Hi all,
> 
> It looks like we have very special "command" character here… which mirrors 
> some "visible" images… but it is still invalid XML when I try to validate in 
> Eclipse… 
> Solr-4.0.0-BETA
> 
> 
> 
> 
> 
>   
>   0
>   237
>   
>   true
>   1
>   index
>   10
>   enrich_keywords_string_mv
>   
>   
>  name="response"
>   numFound="0"
>   start="0"
>   >
>   
>   
>   
>   
>   0
>   0
>   0
>   0
>   0
>   0
>   0
>   0
>   0
>   0
>   
>   
>   
>   
>   
> 

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3138 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3138/

All tests passed

Build Log:
[...truncated 7455 lines...]
[junit4:junit4] ERROR: JVM J0 ended with an exception, command line: 
/usr/local/openjdk7/jre/bin/java -XX:+UseG1GC -Dtests.prefix=tests 
-Dtests.seed=66464D823D2CE41A -Xmx512M -Dtests.iters= -Dtests.verbose=false 
-Dtests.infostream=false 
-Dtests.lockdir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build
 -Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random 
-Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 
-Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/testlogging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true 
-Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. 
-Djava.io.tmpdir=. -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 
-Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/commons-codec-1.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/spatial4j-0.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/lib/zookeeper-3.3.6.jar:/

[jira] [Commented] (SOLR-3751) Add defensive checks against errant updates to distrib update processor.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444996#comment-13444996
 ] 

Mark Miller commented on SOLR-3751:
---

When doing a replicate from a leader, we can also check that the leader locally 
thinks he is the leader.

> Add defensive checks against errant updates to distrib update processor.
> 
>
> Key: SOLR-3751
> URL: https://issues.apache.org/jira/browse/SOLR-3751
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
>
> The distrib update processor should do sanity checks on incoming updates - 
> you never know what greedy routers might hang to some packets for while 
> before letting them go.
> We can quickly and easily check a couple things.
> If the update is from a Leader we can check that we don't currently think we 
> are the leader (using a local isLeader state).
> If the update is from a Leader we can check that that leader matches what is 
> in our current ClusterState.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4343:


Attachment: LUCENE-4343.patch

updated patch:
* MockTokenizer's state machine works via an assert-only testpoint.
* fixed PatternTokenizer to not consume in its ctor :)

two more tokenstreams in solr to fix and we are good: PreAnalyzedField and 
TrieTokenizerFactory.

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch, LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3721) Multiple concurrent recoveries of same shard?

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445001#comment-13445001
 ] 

Mark Miller commented on SOLR-3721:
---

I've committed the fix to the issue I mention in the above comment.

> Multiple concurrent recoveries of same shard?
> -
>
> Key: SOLR-3721
> URL: https://issues.apache.org/jira/browse/SOLR-3721
> Project: Solr
>  Issue Type: Bug
>  Components: multicore, SolrCloud
>Affects Versions: 4.0
> Environment: Using our own Solr release based on Apache revision 
> 1355667 from 4.x branch. Our changes to the Solr version is our solutions to 
> TLT-3178 etc., and should have no effect on this issue.
>Reporter: Per Steffensen
>  Labels: concurrency, multicore, recovery, solrcloud
> Fix For: 4.0
>
> Attachments: recovery_in_progress.png, recovery_start_finish.log
>
>
> We run a performance/endurance test on a 7 Solr instance SolrCloud setup and 
> eventually Solrs lose ZK connections and go into recovery. BTW the recovery 
> often does not ever succeed, but we are looking into that. While doing that I 
> noticed that, according to logs, multiple recoveries are in progress at the 
> same time for the same shard. That cannot be intended and I can certainly 
> imagine that it will cause some problems.
> It is just the logs that are wrong, did I make some mistake, or is this a 
> real bug?
> See attached grep from log, grepping only on "Finished recovery" and 
> "Starting recovery" logs.
> {code}
> grep -B 1 "Finished recovery\|Starting recovery" solr9.log solr8.log 
> solr7.log solr6.log solr5.log solr4.log solr3.log solr2.log solr1.log 
> solr0.log > recovery_start_finish.log
> {code}
> It can be hard to get an overview of the log, but I have generated a graph 
> showing (based alone on "Started recovery" and "Finished recovery" logs) how 
> many recoveries are in progress at any time for the different shards. See 
> attached recovery_in_progress.png. The graph is also a little hard to get an 
> overview of (due to the many shards) but it is clear that for several shards 
> there are multiple recoveries going on at the same time, and that several 
> recoveries never succeed.
> Regards, Per Steffensen

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Invalid XML Output (Arabic):

[Robert Muir]

Steve is right. its just using BiDi algorithm to determine the runs
and getting confused by punctuation.

This has nothing to do with lucene/solr.

On Thu, Aug 30, 2012 at 11:05 AM, Steve McKay  wrote:
> Are you sure this isn't an Eclipse issue? The plaintext (quoted-printable)
> source of the  elements looks like this:
>
>  =D9=88=D9=82=D8=A7=D9=84=D8=AA =D9=8A=D8=B1=D9=88=D8=AD">0
>
> AFAIK this is valid XML. There are only a few codepoints not allowed by XML
> 1.0. I don't see a right-to-left mark, so it seems like Mail.app is trying
> to infer a right-to-left mark for the Arabic text and getting it wrong,
> causing the mangled display.
>
> On Aug 29, 2012, at 2:17 PM, Fuad Efendi  wrote:
>
> Hi all,
>
> It looks like we have very special "command" character here… which mirrors
> some "visible" images… but it is still invalid XML when I try to validate in
> Eclipse…
> Solr-4.0.0-BETA
>
>
>
> 
> 
> 
> 0
> 237
> 
> true
> 1
> index
> 10
> enrich_keywords_string_mv
> 
> 
>  name="response"
> numFound="0"
> start="0"
>>
> 
> 
> 
> 
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 
> 
> 
> 
> 
> 
>
>



-- 
lucidworks.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3772) On cluster startup, we should wait until we see all registered replicas before running the leader process - or if they all do not come up, N amount of time.

[Mark Miller (JIRA)]

Mark Miller created SOLR-3772:
-

 Summary: On cluster startup, we should wait until we see all 
registered replicas before running the leader process - or if they all do not 
come up, N amount of time.
 Key: SOLR-3772
 URL: https://issues.apache.org/jira/browse/SOLR-3772
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.0, 5.0


This seems like a simple and good way to start.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Steven A Rowe]

There is a strange failure here in collecting test results (excerpted below) - 
parsing of XML test output fails because there are missing closing angle 
brackets on the last few elements in the file 
:

114:
115:

Steve

-Original Message-
From: Policeman Jenkins Server [mailto:jenk...@sd-datasolutions.de] 
Sent: Thursday, August 30, 2012 10:57 AM
To: dev@lucene.apache.org; markrmil...@apache.org
Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - 
Failure!

[...]

Recording test results
ERROR: Failed to archive test reports
hudson.util.IOException2: Failed to read 
/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
at hudson.tasks.junit.TestResult.parse(TestResult.java:244)
at hudson.tasks.junit.TestResult.parse(TestResult.java:163)
at hudson.tasks.junit.TestResult.parse(TestResult.java:140)
at hudson.tasks.junit.TestResult.(TestResult.java:116)
at 
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
at 
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
at hudson.FilePath.act(FilePath.java:842)
at hudson.FilePath.act(FilePath.java:824)
at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
at 
hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:122)
at 
hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:134)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)
at hudson.model.Build$BuildExecution.post2(Build.java:183)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)
at hudson.model.Run.execute(Run.java:1527)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: org.dom4j.DocumentException: Error on line 115 of document 
file:///var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
 : Element type "testcase" must be followed by either attribute specifications, 
">" or "/>". Nested exception: Element type "testcase" must be followed by 
either attribute specifications, ">" or "/>".
at org.dom4j.io.SAXReader.read(SAXReader.java:482)
at org.dom4j.io.SAXReader.read(SAXReader.java:264)
at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:112)
at hudson.tasks.junit.TestResult.parse(TestResult.java:227)
... 19 more
Caused by: org.xml.sax.SAXParseException: Element type "testcase" must be 
followed by either attribute specifications, ">" or "/>".
at 
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
at 
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
at 
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
at 
com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1427)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.seekCloseOfStartTag(XMLDocumentFragmentScannerImpl.java:1389)
at 
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:285)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2756)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
at 
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at 
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at 
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at 
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java

[jira] [Commented] (SOLR-3750) On session expiration, we should explicitly wait some time before running the leader sync process so that we are sure every node participates.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445007#comment-13445007
 ] 

Mark Miller commented on SOLR-3750:
---

I think initially we should wait for the known amount of replicas to show up, 
else a set amount of time (N).

We need something like this for a start, but it seems it could be improved. The 
ugly case seems to be when 2 or more nodes go down and are not coming back 
anytime soon...updates would then be down for N amount of time.

> On session expiration, we should explicitly wait some time before running the 
> leader sync process so that we are sure every node participates.
> --
>
> Key: SOLR-3750
> URL: https://issues.apache.org/jira/browse/SOLR-3750
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> We should wait until all the known nodes are part of the election, or X 
> amount of time if that does not happen (when a node or more does not come 
> back for whatever reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3699) SolrIndexWriter constructor leaks Directory if Exception creating IndexWriterConfig

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445009#comment-13445009
 ] 

Hoss Man commented on SOLR-3699:


bq. the setDirFactory method seems a little troubling - why do we do that 
instead of making it part of the constructor? What if you don't set it?

Are you asking about SolrIndexWriter.setDirectoryFactory ?

it's private, and called from the (static) SolrIndexWriter.create -- which as 
of this patch is the only way to get a new SolrIndexWRiter.

I had to move it out of the SolrIndexWriter constructor, because we need to be 
able to pass the resulting directory to super() while also releasing the 
directory if the constructor throws an exception -- this isn't possible within 
java's "calls to super(...) must be the first line of the constructor" 
constraints, so the static factory method was needed instead.

bq. The dir factory should and will be closed by the DefaultSolrCoreState when 
it's ref count hits 0 - you don't actually want to close it when closing a core.

But the update handler is the only thing keeping track of the SolrCoreState -- 
if the SolrCore has no UpdateHanlder (ie: because it failed to initialize) then 
*something* needs to close the DirectoryFactory that 
SolrCore.initDirectoryFactory() already created.

(Go ahead and try to comment out that "HACK" directoryFactory.close() in the 
patch -- you'll see TestBadConfig.testUpdateLogButNoVersionField start failing. 
 I added that call to ensure that the DirectoryFactory gets closed if any 
exceptions are thrown between a (successsful) call to initDirectoryFactory() 
~L641 of SolrCore and (a success of failure of) createUpdateHandler() ~L707.  
Don't get me wrong, it definitely feels like a hack (hence the comment) but 
_something_ needs to account for this situation -- and i wasn't comfortably 
completely re-writing/re-ordering the logic in the SolrCore constructor.



> SolrIndexWriter constructor leaks Directory if Exception creating 
> IndexWriterConfig
> ---
>
> Key: SOLR-3699
> URL: https://issues.apache.org/jira/browse/SOLR-3699
> Project: Solr
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-3699.patch, SOLR-3699.patch, SOLR-3699.patch, 
> SOLR-3699.patch
>
>
> in LUCENE-4278 i had to add a hack to force SimpleFSDir for 
> CoreContainerCoreInitFailuresTest, because it doesnt close its Directory on 
> certain errors.
> This might indicate a problem that leaks happen if certain errors happen 
> (e.g. not handled in finally)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3750) On session expiration, we should explicitly wait some time before running the leader sync process so that we are sure every node participates.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445011#comment-13445011
 ] 

Mark Miller commented on SOLR-3750:
---

To clarify, the above comment is more in regards to the leader goes down case. 
In the connectionloss case, usually everyone should come right back in the 
normal case. When a leader goes down, we can look for numReplicas-1 - but if 2 
or more nodes go down, we would have to wait N amount of time.

> On session expiration, we should explicitly wait some time before running the 
> leader sync process so that we are sure every node participates.
> --
>
> Key: SOLR-3750
> URL: https://issues.apache.org/jira/browse/SOLR-3750
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> We should wait until all the known nodes are part of the election, or X 
> amount of time if that does not happen (when a node or more does not come 
> back for whatever reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nagaraj Molala updated SOLR-3771:
-

Description: 
we are getting below error. Please give us the solution as this is a show 
stopper for our application. Attached the file for your reference.
 
https://issues.apache.org/jira/browse/SOLR 2:51 PM 
Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at sun.net.NetworkClient.doConnect(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
   at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
   at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
   at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
e.java:97)
   ... 13 more

 


  was:
we are getting below error. Please give us the solution as this is a show 
stopper for our application.
 
https://issues.apache.org/jira/browse/SOLR 2:51 PM 
Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at sun.net.NetworkClient.doConnect(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
   at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
   at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
   at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
e.java:97)
   ... 13 more

 



> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.1
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the file for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nagaraj Molala updated SOLR-3771:
-

Affects Version/s: (was: 3.1)
   3.6

> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.6
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the file for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nagaraj Molala updated SOLR-3771:
-

Attachment: rss-data-config.xml
solrconfig.xml
schema.xml

> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.6
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
> Attachments: rss-data-config.xml, schema.xml, solrconfig.xml
>
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the file for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nagaraj Molala updated SOLR-3771:
-

Description: 
we are getting below error. Please give us the solution as this is a show 
stopper for our application. Attached the config files for your reference.
 
https://issues.apache.org/jira/browse/SOLR 2:51 PM 
Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at sun.net.NetworkClient.doConnect(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
   at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
   at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
   at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
e.java:97)
   ... 13 more

 


  was:
we are getting below error. Please give us the solution as this is a show 
stopper for our application. Attached the file for your reference.
 
https://issues.apache.org/jira/browse/SOLR 2:51 PM 
Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
   at java.net.PlainSocketImpl.connect(Unknown Source)
   at java.net.SocksSocketImpl.connect(Unknown Source)
   at java.net.Socket.connect(Unknown Source)
   at sun.net.NetworkClient.doConnect(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.openServer(Unknown Source)
   at sun.net.www.http.HttpClient.(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.http.HttpClient.New(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
   at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
   at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
   at org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
e.java:97)
   ... 13 more

 



> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.6
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
> Attachments: rss-data-config.xml, schema.xml, solrconfig.xml
>
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the config files for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: d

[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #77: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/77/

No tests ran.

Build Log:
[...truncated 1141 lines...]
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected 
termination of the channel
hudson.remoting.RequestAbortedException: 
hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected 
termination of the channel
at hudson.remoting.Request.call(Request.java:174)
at hudson.remoting.Channel.call(Channel.java:663)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy38.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:861)
at hudson.Launcher$ProcStarter.join(Launcher.java:345)
at hudson.tasks.Maven.perform(Maven.java:263)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1502)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: 
Unexpected termination of the channel
at hudson.remoting.Request.abort(Request.java:299)
at hudson.remoting.Channel.terminate(Channel.java:719)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at 
java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2571)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1315)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at hudson.remoting.Command.readFrom(Command.java:90)
at 
hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4226) Efficient compression of small to medium stored fields

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445022#comment-13445022
 ] 

David Smiley commented on LUCENE-4226:
--

I just have a word of encouragement -- this is awesome!  Keep up the good work 
Adrien.

> Efficient compression of small to medium stored fields
> --
>
> Key: LUCENE-4226
> URL: https://issues.apache.org/jira/browse/LUCENE-4226
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Adrien Grand
>Priority: Trivial
> Attachments: CompressionBenchmark.java, CompressionBenchmark.java, 
> LUCENE-4226.patch, LUCENE-4226.patch, SnappyCompressionAlgorithm.java
>
>
> I've been doing some experiments with stored fields lately. It is very common 
> for an index with stored fields enabled to have most of its space used by the 
> .fdt index file. To prevent this .fdt file from growing too much, one option 
> is to compress stored fields. Although compression works rather well for 
> large fields, this is not the case for small fields and the compression ratio 
> can be very close to 100%, even with efficient compression algorithms.
> In order to improve the compression ratio for small fields, I've written a 
> {{StoredFieldsFormat}} that compresses several documents in a single chunk of 
> data. To see how it behaves in terms of document deserialization speed and 
> compression ratio, I've run several tests with different index compression 
> strategies on 100,000 docs from Mike's 1K Wikipedia articles (title and text 
> were indexed and stored):
>  - no compression,
>  - docs compressed with deflate (compression level = 1),
>  - docs compressed with deflate (compression level = 9),
>  - docs compressed with Snappy,
>  - using the compressing {{StoredFieldsFormat}} with deflate (level = 1) and 
> chunks of 6 docs,
>  - using the compressing {{StoredFieldsFormat}} with deflate (level = 9) and 
> chunks of 6 docs,
>  - using the compressing {{StoredFieldsFormat}} with Snappy and chunks of 6 
> docs.
> For those who don't know Snappy, it is compression algorithm from Google 
> which has very high compression ratios, but compresses and decompresses data 
> very quickly.
> {noformat}
> Format   Compression ratio IndexReader.document time
> 
> uncompressed 100%  100%
> doc/deflate 1 59%  616%
> doc/deflate 9 58%  595%
> doc/snappy80%  129%
> index/deflate 1   49%  966%
> index/deflate 9   46%  938%
> index/snappy  65%  264%
> {noformat}
> (doc = doc-level compression, index = index-level compression)
> I find it interesting because it allows to trade speed for space (with 
> deflate, the .fdt file shrinks by a factor of 2, much better than with 
> doc-level compression). One other interesting thing is that {{index/snappy}} 
> is almost as compact as {{doc/deflate}} while it is more than 2x faster at 
> retrieving documents from disk.
> These tests have been done on a hot OS cache, which is the worst case for 
> compressed fields (one can expect better results for formats that have a high 
> compression ratio since they probably require fewer read/write operations 
> from disk).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Moved] (LUCENE-4345) Create a Classification component

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili moved SOLR-3700 to LUCENE-4345:
---

Lucene Fields: New
  Key: LUCENE-4345  (was: SOLR-3700)
  Project: Lucene - Core  (was: Solr)

> Create a Classification component
> -
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4345) Create a Classification component

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445020#comment-13445020
 ] 

Tommaso Teofili commented on LUCENE-4345:
-

ok, now this is moved to Lucene

> Create a Classification component
> -
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-4345:


Summary: Create a Classification module  (was: Create a Classification 
component)

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445029#comment-13445029
 ] 

Hoss Man commented on SOLR-3771:


1) "xx.abcd.abcd.com" from your stack trace is not a valid hostname.
2) your attached rss-data-config.xml has the url scrubed and replaced with 
"company URL comes here .."

There is no evidence of any actual problem other then the one already mentioned 
in the error message form solr: "UnknownHostException" .. ie: the hostname 
specified does not appear to exist.

Without more detials on what host you are trying to connect to, and wether or 
not that that host can be resolved by DNS clients on the machine where you are 
running Solr, there's really no help anyone can offer you.

> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.6
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
> Attachments: rss-data-config.xml, schema.xml, solrconfig.xml
>
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the config files for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4343:


Attachment: LUCENE-4343.patch

Updated patch: with setReader as final :)

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch, LUCENE-4343.patch, LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Dawid Weiss]

I will investigate this later today. Thanks Steve.

Sent from mobile phone.
On Aug 30, 2012 5:18 PM, "Steven A Rowe"  wrote:

> There is a strange failure here in collecting test results (excerpted
> below) - parsing of XML test output fails because there are missing closing
> angle brackets on the last few elements in the file <
> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/ws/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml/*view*/
> >:
>
> 114: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides"
> name="testBefore" time="0.011"/>
> 115: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides"
> name="testAfter" time="0.011"/
> 116:
> 117: 118:
> 119: 120:
> 121: 
>
> Steve
>
> -Original Message-
> From: Policeman Jenkins Server [mailto:jenk...@sd-datasolutions.de]
> Sent: Thursday, August 30, 2012 10:57 AM
> To: dev@lucene.apache.org; markrmil...@apache.org
> Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752
> - Failure!
>
> [...]
>
> Recording test results
> ERROR: Failed to archive test reports
> hudson.util.IOException2: Failed to read
> /var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
> at hudson.tasks.junit.TestResult.parse(TestResult.java:244)
> at hudson.tasks.junit.TestResult.parse(TestResult.java:163)
> at hudson.tasks.junit.TestResult.parse(TestResult.java:140)
> at hudson.tasks.junit.TestResult.(TestResult.java:116)
> at
> hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
> at
> hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
> at hudson.FilePath.act(FilePath.java:842)
> at hudson.FilePath.act(FilePath.java:824)
> at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
> at
> hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:122)
> at
> hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:134)
> at
> hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)
> at hudson.model.Build$BuildExecution.post2(Build.java:183)
> at
> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)
> at hudson.model.Run.execute(Run.java:1527)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
> at
> hudson.model.ResourceController.execute(ResourceController.java:88)
> at hudson.model.Executor.run(Executor.java:236)
> Caused by: org.dom4j.DocumentException: Error on line 115 of document
> file:///var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
> : Element type "testcase" must be followed by either attribute
> specifications, ">" or "/>". Nested exception: Element type "testcase" must
> be followed by either attribute specifications, ">" or "/>".
> at org.dom4j.io.SAXReader.read(SAXReader.java:482)
> at org.dom4j.io.SAXReader.read(SAXReader.java:264)
> at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:112)
> at hudson.tasks.junit.TestResult.parse(TestResult.java:227)
> ... 19 more
> Caused by: org.xml.sax.SAXParseException: Element type "testcase" must be
> followed by either attribute specifications, ">" or "/>".
> at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> at
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> at
> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1427)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.seekCloseOfStartTag(XMLDocumentFragmentScannerImpl.java:1389)
> at
> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:285)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2756)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
> at
> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:51

[jira] [Commented] (SOLR-3771) While using RSS indexing from Solr, we are getting error "Caused by: java.net.UnknownHostException" & indexing fail.

[Nagaraj Molala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445040#comment-13445040
 ] 

Nagaraj Molala commented on SOLR-3771:
--

Hi Hoss,

For security reason i did not disclose the client URL here. I know 
"xx.abcd.abcd.com" is not a valid host. Could you please let me know the steps 
for setting the proxy at my end? 

OS - Windows 7.0
Solr - 3.6

Nagaraj

> While using RSS indexing from Solr, we are getting error "Caused by: 
> java.net.UnknownHostException" & indexing fail.
> 
>
> Key: SOLR-3771
> URL: https://issues.apache.org/jira/browse/SOLR-3771
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 3.6
> Environment: Solr Search engine using RSS data feeding
>Reporter: Nagaraj Molala
> Attachments: rss-data-config.xml, schema.xml, solrconfig.xml
>
>
> we are getting below error. Please give us the solution as this is a show 
> stopper for our application. Attached the config files for your reference.
>  
> https://issues.apache.org/jira/browse/SOLR 2:51 PM 
> Caused by: java.net.UnknownHostException: xx.abcd.abcd.com
>at java.net.PlainSocketImpl.connect(Unknown Source)
>at java.net.SocksSocketImpl.connect(Unknown Source)
>at java.net.Socket.connect(Unknown Source)
>at sun.net.NetworkClient.doConnect(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.openServer(Unknown Source)
>at sun.net.www.http.HttpClient.(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.http.HttpClient.New(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
> Source)
>at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown 
> Sour
> ce)
>at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
>at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown 
> So
> urce)
>at 
> org.apache.solr.handler.dataimport.URLDataSource.getData(URLDataSourc
> e.java:97)
>... 13 more
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3699) SolrIndexWriter constructor leaks Directory if Exception creating IndexWriterConfig

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445047#comment-13445047
 ] 

Mark Miller commented on SOLR-3699:
---

bq. it's private,

Ah, okay, did not catch that.

bq. something needs to close the DirectoryFactory that 
SolrCore.initDirectoryFactory() already created.

Yeah, brain murmur - for some reason I was thinking DirectoryFactory was used 
globally across cores.

This is probably fine then. Once I commit some stuff from my workspace, I'll 
actually apply the patch and try and consider this some more.

> SolrIndexWriter constructor leaks Directory if Exception creating 
> IndexWriterConfig
> ---
>
> Key: SOLR-3699
> URL: https://issues.apache.org/jira/browse/SOLR-3699
> Project: Solr
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-3699.patch, SOLR-3699.patch, SOLR-3699.patch, 
> SOLR-3699.patch
>
>
> in LUCENE-4278 i had to add a hack to force SimpleFSDir for 
> CoreContainerCoreInitFailuresTest, because it doesnt close its Directory on 
> certain errors.
> This might indicate a problem that leaks happen if certain errors happen 
> (e.g. not handled in finally)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4339) Allow deletions on Lucene3x codecs again

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445048#comment-13445048
 ] 

Michael McCandless commented on LUCENE-4339:


Yes this bug is pre-existing ... but I think it doesn't break anything, ie, 
it's a performance bug since on every commit it needlessly goes and writes 
_N.si files again.

> Allow deletions on Lucene3x codecs again
> 
>
> Key: LUCENE-4339
> URL: https://issues.apache.org/jira/browse/LUCENE-4339
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Priority: Blocker
> Fix For: 4.0
>
> Attachments: LUCENE-4339.patch, LUCENE-4339.patch
>
>
> On dev@lao Hoss reported that a user in Solr was not able to update or delete 
> documents in his 3.x index with Solr 4:
> {quote}
> On the solr-user list, Dirk Högemann recently mentioned a problem he was 
> seeing when he tried upgrading his existing solr setup from 3.x to 4.0-BETA.  
> Specifically this exception getting logged...
> http://find.searchhub.org/document/cdb30099bfea30c6
> auto commit error...:java.lang.UnsupportedOperationException: this codec can 
> only be used for reading
>  at 
> org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.writeLiveDocs(Lucene3xCodec.java:74)
>  at 
> org.apache.lucene.index.ReadersAndLiveDocs.writeLiveDocs(ReadersAndLiveDocs.java:278)
>  at 
> org.apache.lucene.index.IndexWriter$ReaderPool.release(IndexWriter.java:435)
>  at 
> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:278)
>  at 
> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2928)
>  at 
> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2919)
>  at 
> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2666)
>  at 
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2793)
>  at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2773)
>  at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:531)
>  at org.apache.solr.update.CommitTracker.run(CommitTracker.java:214)
> Dirk was able to work arround this by completely re-indexing, but it seemed 
> strange to me that this would happen.
> My understanding is that even though an IndexUpgrader tool was now available, 
> it wasn't going to be required for users to use it when upgrading from 3.x to 
> 4.x.  Explicitly upgrading the index format might be a good idea, and might 
> make hte index more performant, but as I understood it, the way things had 
> been implemented with codecs explicitly upgrading the index format wasn't 
> strictly neccessary, and that users should be able to upgrade their lucene 
> apps same way that was supported with other index format upgrades in the 
> past: the old index can be read, and as changes are made new segments will be 
> re-written in the new format.  (Note in
> particular: at the moment we don't mention IndexUpgrader in MIGRATE.txt at 
> all.)
> It appears however, based on this stack trace and some other experiements i 
> tried, that any attempts to "delete" documents in a segment that is using the 
> Lucene3xCodec will fail.
> This seems like a really scary time bomb sitaution, because if you upgrade, 
> things will seem to be working -- you can even add documents, and depending 
> on the order that you do things, some "old" segments may get merged and use 
> the new format, so *some* deletes of "old" documents (in those 
> merged/upgraded) segments may work, but then somewhere down the road, you may 
> try to a delete that affects docs in a still un-merge/upgraded segment, and 
> that delete will fail -- 5 minutes later, if another merge has happened, 
> attempting to do the exact same delete may succeed.
> All of which begs the question: is this a known/intended limitation of the 
> Lucene3xCodec, or an oversight in the Lucene3xCodec?
> if it's expected, then it seems like we should definitely spell out this 
> limitation in MIGRATE.txt and advocate either full rebuilds, or the use of 
> IndexUpgrader for anyone who's indexes are non-static.
> On the Solr side of things, i think we should even want to consider 
> automaticly running IndexUpgrader on startup if we detect that the 
> Lucene3xCodec is in use to simplify things -- we can't even suggest running 
> "optimize" as a quick/easy way to force and index format upgrade because if 
> the 3x index as already optimized then it's a no-op and the index stays in 
> the 3x format.
> {quote}
> Robert said, that this is a wanted limitation (in fact its explicitely added 
> to the code, without that UOE it "s

[JENKINS] Lucene-Solr-Tests-4.x-Java6 - Build # 556 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/556/

All tests passed

Build Log:
[...truncated 6138 lines...]
[junit4:junit4] ERROR: JVM J0 ended with an exception, command line: 
/usr/local/openjdk6/jre/bin/java -Dtests.prefix=tests 
-Dtests.seed=CD9F2451CFB39A89 -Xmx512M -Dtests.iters= -Dtests.verbose=false 
-Dtests.infostream=false 
-Dtests.lockdir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build
 -Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random 
-Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 
-Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/testlogging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true 
-Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. 
-Djava.io.tmpdir=. 
-Dtests.sandbox.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core
 -Djava.security.manager=java.lang.SecurityManager 
-Djava.security.policy=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/tools/junit4/tests.policy
 -Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Dfile.encoding=ISO-8859-1 -classpath 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/analysis/common/lucene-analyzers-common-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/highlighter/lucene-highlighter-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/memory/lucene-memory-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/misc/lucene-misc-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/spatial/lucene-spatial-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/suggest/lucene-suggest-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/grouping/lucene-grouping-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/queries/lucene-queries-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/build/queryparser/lucene-queryparser-4.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/commons-codec-1.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Jav

[jira] [Commented] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445053#comment-13445053
 ] 

Michael McCandless commented on LUCENE-4343:


bq. Updated patch: with setReader as final

Yay!  Patch looks great.  +1

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch, LUCENE-4343.patch, LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-4343:


Attachment: LUCENE-4343.patch

Just a tiny update to make PreAnalyzedTokenizer more correct.

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Attachments: LUCENE-4343.patch, LUCENE-4343.patch, LUCENE-4343.patch, 
> LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3730) Rollback is not implemented quite right and can cause corner case fails in SolrCloud tests.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3730.
---

Resolution: Fixed

> Rollback is not implemented quite right and can cause corner case fails in 
> SolrCloud tests.
> ---
>
> Key: SOLR-3730
> URL: https://issues.apache.org/jira/browse/SOLR-3730
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
>
> Rollback was calling a rollback and then close rather than just a rollback - 
> an oversight on my part - somehow this allowed two indexwriters to be open at 
> once in some seemingly rare cases...this could cause havoc and was easy to 
> miss due to tests using the single lock type.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3768) add some prelim assertions to OverseerTest

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-3768.


   Resolution: Fixed
Fix Version/s: 5.0
   4.0
 Assignee: Hoss Man

Committed revision 1379014.
Committed revision 1379016.


> add some prelim assertions to OverseerTest
> --
>
> Key: SOLR-3768
> URL: https://issues.apache.org/jira/browse/SOLR-3768
> Project: Solr
>  Issue Type: Test
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 4.0, 5.0
>
> Attachments: capture-1.jpg, SOLR-3768.patch
>
>
> There isn't much i understand about OverseerTest, but today while doing a 
> full test run I got an unreproducible assertion failure from this line...
> {noformat}
> assertNotNull("could not find counter for shard:" + ids[i], ai);
> {noformat}
> ...in which the assertion message indicated that not only was "ai" null, but 
> "ids[i]" was null as well.
> Poking arround the test a bit, i think what's happening here is that some of 
> the preliminary logic in testShardAssignmentBigger has bounded wait loops to 
> "make sure " things have happened, but there is no assertion that these 
> things actually happen if that the loop bound is exhausted - which can lead 
> to missleading/confusing errors further on in the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3658) SolrCmdDistributor can briefly create spikes of threads in the thousands.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3658.
---

   Resolution: Fixed
Fix Version/s: 5.0

> SolrCmdDistributor can briefly create spikes of threads in the thousands.
> -
>
> Key: SOLR-3658
> URL: https://issues.apache.org/jira/browse/SOLR-3658
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3658.patch
>
>
> see mailing list http://markmail.org/thread/yy5b7g6g7733wgcp

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3773) Hash based on the external String id rather than the indexed representation for distributed updates.

[Mark Miller (JIRA)]

Mark Miller created SOLR-3773:
-

 Summary: Hash based on the external String id rather than the 
indexed representation for distributed updates.
 Key: SOLR-3773
 URL: https://issues.apache.org/jira/browse/SOLR-3773
 Project: Solr
  Issue Type: Improvement
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.0, 5.0


This makes some things easier - such as external systems doing the hashing 
themselves (like CloudSolrServer).

It will be back compat for String id types but not others. Sent an email to the 
list a good time back seeing if this would bother anyone too much and got no 
replies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3699) SolrIndexWriter constructor leaks Directory if Exception creating IndexWriterConfig

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445143#comment-13445143
 ] 

Hoss Man commented on SOLR-3699:


bq. Yeah, brain murmur - for some reason I was thinking DirectoryFactory was 
used globally across cores.

I think it's only shared between reloaded versions of the same SolrCore (ie: 
two differnet SolrCore's can use completley diff DirectoryFactory impls) but 
even then, it's only shared if the SolrCore was constructed using an existing 
UpdateHandler (ie: reload) if the SolrCore is the "first" SolrCore, then it 
creates the DirectoryFactory, thus if the UpdateHandler doesn't init properly, 
that SolrCore needs to close the DirectoryFactory.

Ideally, a whole metric-shit-load of the SolrCore initialization would be 
reworked, so thta every type of object (like DirectoryFactory) was only ever 
inited by one other object (like UpdateHandler) regardless of wether it's the 
first create or reload state, ... but i wasn't really comfortable making that 
heavy of a change.


> SolrIndexWriter constructor leaks Directory if Exception creating 
> IndexWriterConfig
> ---
>
> Key: SOLR-3699
> URL: https://issues.apache.org/jira/browse/SOLR-3699
> Project: Solr
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-3699.patch, SOLR-3699.patch, SOLR-3699.patch, 
> SOLR-3699.patch
>
>
> in LUCENE-4278 i had to add a hack to force SimpleFSDir for 
> CoreContainerCoreInitFailuresTest, because it doesnt close its Directory on 
> certain errors.
> This might indicate a problem that leaks happen if certain errors happen 
> (e.g. not handled in finally)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: large messages from Jenkins failures

[Steven A Rowe]

The Groovy-scripted email length reduction method appears to be working now 
(after quite a few false starts).

I've added the following script, which limits email body text to 200K chars, to 
all Lucene and Solr jobs on all three Jenkins instances that regularly send 
failure emails to this list (ASF, SDDS, and flonkings):

maxLength = 20;
trailingLength = 1;
bodyPart = msg.getContent().getBodyPart(0);
body = bodyPart.getContent();
bodyLength = body.length();
logger.println("[Email-ext] Notification email body length: " + 
bodyLength);
if (bodyLength > maxLength) {
  text = new StringBuilder();
  text.append(body.substring(0, maxLength - trailingLength));
  text.append("\n\n[...truncated too long message...]\n\n");
  text.append(body.substring(bodyLength - trailingLength));
  bodyPart.setText(text.toString(), "UTF-8");
  logger.println("[Email-ext] Reduced notification email body length 
to: " + text.length());
}

You can see the first successfully length-reduced email to this list, sent from 
Policeman Jenkins Server a little less than an hour before this post, with 
subject "[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_06) - Build # 533 
- Failure!".

I chose the 200K chars limit somewhat arbitrarily - please let me know if you 
think it should be different.

Steve

-Original Message-
From: Steven A Rowe [mailto:sar...@syr.edu] 
Sent: Tuesday, August 28, 2012 3:56 PM
To: dev@lucene.apache.org
Subject: RE: large messages from Jenkins failures

Actually, after discussing with Uwe on #lucene-dev IRC, I'm looking into 
another mechanism to reduce the size of email messages: the Jenkins Email-Ext 
plugin has a per-build-job configuration item named "Pre-send script" that 
allows you to modify the MimeMessage object representing an email via a Groovy 
script.  Here's what I've got so far - I'm going to enable this now on all the 
jobs on Uwe's Jenkins (the "msg" variable, of type MimeMessage, is made 
available by the plugin to the script):

maxLength = 20;
trailingLength = 1;
content = msg.getContent(); // assumption: mime type is "text/plain"
contentLength = content.length();
if (content.length() > maxLength) {
  text = content.substring(0, maxLength - trailingLength) 
   + "\n\n[... truncated too long message ...]\n\n"
   + content.substring(contentLength - trailingLength);
  msg.setText(text, "UTF-8");
}

Steve

-Original Message-
From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley
Sent: Tuesday, August 28, 2012 3:11 PM
To: dev@lucene.apache.org
Subject: Re: large messages from Jenkins failures

On Mon, Aug 20, 2012 at 2:22 PM, Dawid Weiss
 wrote:
> Oh, one more thing -- if we suppress the console output we would
> absolutely have to keep (at jenkins) multiple tests-report.txt files
> because these always contain full output dumps (regardless of console
> settings). Otherwise we'd suppress potentially important info.

+1 to not forward truckloads of info to the mailing lists, as long as
we can easily get at it via jenkins or some other mechanism.

-Yonik
http://lucidworks.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Dawid Weiss]

I'll take a look but it looks like the same cause (something
deadlocks, then an interrupt is sent to all threads and the JVM
crashes). Only seems to happen on freebsd so far?

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Uwe Schindler]

Yes that happens only on FreeBSD.

The other problems were caused by the SecurityManager that were not working as 
expected on lucene.zones.apache.org, because it's network configuration is 
broken (routing table or similar):

If you connect to 127.0.0.1 or localhost, the connection is accepted by the 
listener, but the listener sees not 127.0.0.1 as "remote" address but the 
machine's public IP. This was also checked by me with doing crazy netcat/ssh 
stuff connecting to localhost, but it appears like remote connection. I assume 
that the routing table has a wrong entry that directs 127.0.0.1 through the 
public networking adaptor. I will ask Infra what can cause this!

I added a workaround (marked by comment into the policy file) and also added 
other possible misconfigurations (like localhost resolves not to 127.0.0.1 or 
::1 on certain machines).

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of
> Dawid Weiss
> Sent: Thursday, August 30, 2012 8:12 PM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure
> 
> I'll take a look but it looks like the same cause (something deadlocks, then 
> an
> interrupt is sent to all threads and the JVM crashes). Only seems to happen on
> freebsd so far?
> 
> Dawid
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3773) Hash based on the external String id rather than the indexed representation for distributed updates.

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445166#comment-13445166
 ] 

Yonik Seeley commented on SOLR-3773:


I'd go as far as to strongly recommend people use string as the type of their 
ID field.
Some of the ways I've thought about doing cloud in conjunction with nested 
documents and document blocks involves encoding sub-children as 

The original idea around letting people specify their own uniqueKey field and 
fieldType was that you could design a Solr schema that would be compatible with 
a random lucene index.  I think that's rapidly becoming an outdated idea, and 
not worth the issues it causes.

> Hash based on the external String id rather than the indexed representation 
> for distributed updates.
> 
>
> Key: SOLR-3773
> URL: https://issues.apache.org/jira/browse/SOLR-3773
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> This makes some things easier - such as external systems doing the hashing 
> themselves (like CloudSolrServer).
> It will be back compat for String id types but not others. Sent an email to 
> the list a good time back seeing if this would bother anyone too much and got 
> no replies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Dawid Weiss]

> If you connect to 127.0.0.1 or localhost, the connection is accepted by the 
> listener, but the listener sees not 127.0.0.1 as "remote" address but the 
> machine's public IP. This was also checked by me with doing crazy netcat/ssh 
> stuff connecting to localhost, but it appears like remote connection. I 
> assume that the routing table has a wrong entry that directs 127.0.0.1 
> through the public networking adaptor. I will ask Infra what can cause this!

Do you think that stall on a network socket might have been caused by this, Uwe?

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Uwe Schindler]

The JVM crash? I think yes, because testDistribSearch drove also crazy, because 
connections were only causing SecurityException.

Uwe



Dawid Weiss  schrieb:

>> If you connect to 127.0.0.1 or localhost, the connection is accepted
>by the listener, but the listener sees not 127.0.0.1 as "remote"
>address but the machine's public IP. This was also checked by me with
>doing crazy netcat/ssh stuff connecting to localhost, but it appears
>like remote connection. I assume that the routing table has a wrong
>entry that directs 127.0.0.1 through the public networking adaptor. I
>will ask Infra what can cause this!
>
>Do you think that stall on a network socket might have been caused by
>this, Uwe?
>
>Dawid
>
>-
>To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>For additional commands, e-mail: dev-h...@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de

[jira] [Commented] (SOLR-3765) Wrong handling of documents with same id in cross collection searches

[Per Steffensen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445210#comment-13445210
 ] 

Per Steffensen commented on SOLR-3765:
--

No problem. Glad to help. 

We will not be working on a fix. We will do a workaround in our own 
application, so that we will not have id-clash across collections. We need to 
control ids very strictly in order for our 
fail-on-unique-key-constraint-violaton to serve its purpose correctly. 
Basically we just prefix our ids with the name of the collection - will still 
provide unique-key-clash within the collection but will not prevent documents 
with same id (except for the collection-name-part) from being returned/counted.

> Wrong handling of documents with same id in cross collection searches
> -
>
> Key: SOLR-3765
> URL: https://issues.apache.org/jira/browse/SOLR-3765
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0
> Environment: Self-build version of Solr fra 4.x branch (revision )
>Reporter: Per Steffensen
>  Labels: collections, inconsistency, numFound, search
>
> Dialog with myself from solr-users mailing list:
> Per Steffensen skrev:
> {quote} 
> Hi
> Due to what we have seen in recent tests I got in doubt how Solr search is 
> actually supposed to behave
> * Searching with "distrib=true&q=*:*&rows=10&collection=x,y,z&sort=timestamp 
> asc"
> ** Is Solr supposed to return the 10 documents with the lowest timestamp 
> across all documents in all slices of collection x, y and z, or is it 
> supposed to just pick 10 random documents from those slices and just sort 
> those 10 randomly selected documents?
> ** Put in another way - is this search supposed to be consistent, returning 
> exactly the same set of documents when performed several times (no documents 
> are updated between consecutive searches)?
> {quote}
> Fortunately I believe the answer is, that it ought to "return the 10 
> documents with the lowest timestamp across all documents in all slices of 
> collection x, y and Z". The reason I asked was because I got different 
> responses for consecutive simular requests. Now I believe it can be explained 
> by the bug described below. I guess they you do cross-collection/shard 
> searches, the "request-handling" Solr forwards the query to all involved 
> shards simultanious and merges sub-results into the final result as they are 
> returned from the shards. Because of the "consider documents with same id as 
> the same document even though the come from different collections"-bug it is 
> kinda random (depending on which shards responds first/last), for a given id, 
> what collection the document with that specific id is taken from. And if 
> documents with the same id from different collections has different timestamp 
> it is random where that document ends up in the final sorted result.
> So i believe this inconsistency can be explained by the bug described below.
> {quote}
> * A search returns a "numFound"-field telling how many documents all in all 
> matches the search-criteria, even though not all those documents are returned 
> by the search. It is a crazy question to ask, but I will do it anyway because 
> we actually see a problem with this. Isnt it correct that two searches which 
> only differs on the "rows"-number (documents to be returned) should always 
> return the same value for "numFound"?
> {quote}
> Well I found out myself what the problem is (or seems to be) - see:
> http://lucene.472066.n3.nabble.com/Changing-value-of-start-parameter-affects-numFound-td2460645.html
> http://lucene.472066.n3.nabble.com/numFound-inconsistent-for-different-rows-param-td3997269.html
> http://lucene.472066.n3.nabble.com/Solr-v3-5-0-numFound-changes-when-paging-through-results-on-8-shard-cluster-td3990400.html
> Until 4.0 this "bug" could be "ignored" because it was ok for a cross-shards 
> search to consider documents with identical id's as dublets and therefore 
> only returning/counting one of them. It is still, in 4.0, ok within the same 
> collection, but across collections identical id's should not be considered 
> dublicates and should not reduce documents returned/counted. So i believe 
> this "feature" has now become a bug in 4.0 when it comes to cross-collections 
> searches.
> {quote}
> Thanks!
> Regards, Steff
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4343) Clear up more Tokenizer.setReader/TokenStream.reset issues

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-4343.
-

   Resolution: Fixed
Fix Version/s: 4.0
   5.0

> Clear up more Tokenizer.setReader/TokenStream.reset issues
> --
>
> Key: LUCENE-4343
> URL: https://issues.apache.org/jira/browse/LUCENE-4343
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/analysis
>Reporter: Robert Muir
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4343.patch, LUCENE-4343.patch, LUCENE-4343.patch, 
> LUCENE-4343.patch
>
>
> spinoff from user-list thread.
> I think the rename helps, but the javadocs still have problems: they seem to 
> only describe a totally wacky case (CachingTokenFilter) and not the normal 
> case.
> Ideally setReader would be final I think, but there are a few crazy 
> tokenstreams to fix before I could make that work. Would also need something 
> hackish so MockTokenizer's state machine is still functional.
> But i worked on fixing up the mess in our various tokenstreams, which is easy 
> for the most part.
> As part of this I found it was really useful in flushing out test bugs (ones 
> that dont use MockTokenizer, which they really should), if we can do some 
> best-effort exceptions when the consumer is broken and it costs nothing.
> For example:
> {noformat}
> -  private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
> +  // note: bufferIndex is -1 here to best-effort AIOOBE consumers that don't 
> call reset()
> +  private int offset = 0, bufferIndex = -1, dataLen = 0, finalOffset = 0;
> {noformat}
> I think this is worth exploring more... this was really effective at finding 
> broken tests etc. We should see if we can be more thorough/ideally throw 
> better exceptions when consumers are broken and its free.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Dawid Weiss]

> The JVM crash? I think yes, because testDistribSearch drove also crazy,
> because connections were only causing SecurityException.

Yeah, but I mean before you committed the security policies -- the JVM
crashes were a result of a test thread waiting on a socket read()
indefinitely (until it was interrupted by a timeout). The interrupt
was typically ending well but soon after that the JVM was crashing.
Weird.

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3710) Change CloudSolrServer so that update requests are first sent to leaders by default.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3710:
--

Fix Version/s: 5.0
  Summary: Change CloudSolrServer so that update requests are first 
sent to leaders by default.  (was: Change CloudSolrServer so that update 
requests are only sent to leaders by default.)

> Change CloudSolrServer so that update requests are first sent to leaders by 
> default.
> 
>
> Key: SOLR-3710
> URL: https://issues.apache.org/jira/browse/SOLR-3710
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3710.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3710) Change CloudSolrServer so that update requests are first sent to leaders by default.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3710.
---

Resolution: Fixed

> Change CloudSolrServer so that update requests are first sent to leaders by 
> default.
> 
>
> Key: SOLR-3710
> URL: https://issues.apache.org/jira/browse/SOLR-3710
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3710.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3752) Currently it seems that when a leader goes down, he stays marked as the leader in the ClusterState - it's not strictly necessary, but I think the leader should be cleared

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3752.
---

Resolution: Fixed

> Currently it seems that when a leader goes down, he stays marked as the 
> leader in the ClusterState - it's not strictly necessary, but I think the 
> leader should be cleared until the new one is elected.
> 
>
> Key: SOLR-3752
> URL: https://issues.apache.org/jira/browse/SOLR-3752
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Trivial
> Fix For: 4.0, 5.0
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3751) Add defensive checks against errant updates to distrib update processor.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-3751.
---

Resolution: Fixed

> Add defensive checks against errant updates to distrib update processor.
> 
>
> Key: SOLR-3751
> URL: https://issues.apache.org/jira/browse/SOLR-3751
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
>
> The distrib update processor should do sanity checks on incoming updates - 
> you never know what greedy routers might hang to some packets for while 
> before letting them go.
> We can quickly and easily check a couple things.
> If the update is from a Leader we can check that we don't currently think we 
> are the leader (using a local isLeader state).
> If the update is from a Leader we can check that that leader matches what is 
> in our current ClusterState.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3721) Multiple concurrent recoveries of same shard?

[Per Steffensen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445219#comment-13445219
 ] 

Per Steffensen commented on SOLR-3721:
--

Cool. I dont believe this was our issue, as you also mention. But any fix of a 
potential problem is a step in the right direction.

It will probably be a while before I get back to this issue, because we have 
changed our strategy. Before we where aming at including replication in first 
production-ready version of our application. We have had too many problems with 
the immature (IMHO) new way of doing replication in 4.0. Now we aim at not 
including replication, so I will not be dealing with replication to much for 
the next few months.

> Multiple concurrent recoveries of same shard?
> -
>
> Key: SOLR-3721
> URL: https://issues.apache.org/jira/browse/SOLR-3721
> Project: Solr
>  Issue Type: Bug
>  Components: multicore, SolrCloud
>Affects Versions: 4.0
> Environment: Using our own Solr release based on Apache revision 
> 1355667 from 4.x branch. Our changes to the Solr version is our solutions to 
> TLT-3178 etc., and should have no effect on this issue.
>Reporter: Per Steffensen
>  Labels: concurrency, multicore, recovery, solrcloud
> Fix For: 4.0
>
> Attachments: recovery_in_progress.png, recovery_start_finish.log
>
>
> We run a performance/endurance test on a 7 Solr instance SolrCloud setup and 
> eventually Solrs lose ZK connections and go into recovery. BTW the recovery 
> often does not ever succeed, but we are looking into that. While doing that I 
> noticed that, according to logs, multiple recoveries are in progress at the 
> same time for the same shard. That cannot be intended and I can certainly 
> imagine that it will cause some problems.
> It is just the logs that are wrong, did I make some mistake, or is this a 
> real bug?
> See attached grep from log, grepping only on "Finished recovery" and 
> "Starting recovery" logs.
> {code}
> grep -B 1 "Finished recovery\|Starting recovery" solr9.log solr8.log 
> solr7.log solr6.log solr5.log solr4.log solr3.log solr2.log solr1.log 
> solr0.log > recovery_start_finish.log
> {code}
> It can be hard to get an overview of the log, but I have generated a graph 
> showing (based alone on "Started recovery" and "Finished recovery" logs) how 
> many recoveries are in progress at any time for the different shards. See 
> attached recovery_in_progress.png. The graph is also a little hard to get an 
> overview of (due to the many shards) but it is clear that for several shards 
> there are multiple recoveries going on at the same time, and that several 
> recoveries never succeed.
> Regards, Per Steffensen

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3154) SolrJ CloudServer should be leader and network aware when adding docs

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-3154:
--

Fix Version/s: (was: 4.1)
   5.0
   4.0
 Assignee: Mark Miller

> SolrJ CloudServer should be leader and network aware when adding docs
> -
>
> Key: SOLR-3154
> URL: https://issues.apache.org/jira/browse/SOLR-3154
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 4.0-ALPHA
>Reporter: Grant Ingersoll
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3154.patch
>
>
> It would be good when indexing if the SolrJ CloudServer was leader aware so 
> that we could avoid doing an extra hop for the data.  It would also be good 
> if one could easily set things up based on data locality principles.  This 
> might mean that CloudServer is aware of where on the network it is and would 
> pick leaders that are as close as possible (i.e. local, perhaps.)  This would 
> come in to play when working with tools like Hadoop or other grid computing 
> frameworks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 18 - Failure

[Uwe Schindler]

It works now as before. Policy problem with jail config surrounded by a hack...

Uwe



Dawid Weiss  schrieb:

>> The JVM crash? I think yes, because testDistribSearch drove also
>crazy,
>> because connections were only causing SecurityException.
>
>Yeah, but I mean before you committed the security policies -- the JVM
>crashes were a result of a test thread waiting on a socket read()
>indefinitely (until it was interrupted by a timeout). The interrupt
>was typically ending well but soon after that the JVM was crashing.
>Weird.
>
>Dawid
>
>-
>To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>For additional commands, e-mail: dev-h...@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de

[jira] [Commented] (SOLR-3750) On session expiration, we should explicitly wait some time before running the leader sync process so that we are sure every node participates.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445225#comment-13445225
 ] 

Mark Miller commented on SOLR-3750:
---

I'd like to come up with a goo N to wait - initially I'm going with 5 min. 
Suggestions?

> On session expiration, we should explicitly wait some time before running the 
> leader sync process so that we are sure every node participates.
> --
>
> Key: SOLR-3750
> URL: https://issues.apache.org/jira/browse/SOLR-3750
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> We should wait until all the known nodes are part of the election, or X 
> amount of time if that does not happen (when a node or more does not come 
> back for whatever reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4332) Integrate PiTest mutation coverage tool into build

[Greg Bowyer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445226#comment-13445226
 ] 

Greg Bowyer commented on LUCENE-4332:
-

Do we want to implement this now that Uwe's changes are in 4.0 / trunk ?

> Integrate PiTest mutation coverage tool into build
> --
>
> Key: LUCENE-4332
> URL: https://issues.apache.org/jira/browse/LUCENE-4332
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 4.1, 5.0
>Reporter: Greg Bowyer
>Assignee: Greg Bowyer
>  Labels: build
> Attachments: 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch, 
> LUCENE-4332-Integrate-PiTest-mutation-coverage-tool-into-build.patch
>
>
> As discussed briefly on the mailing list, this patch is an attempt to 
> integrate the PiTest mutation coverage tool into the lucene build

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3750) On session expiration, we should explicitly wait some time before running the leader sync process so that we are sure every node participates.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445241#comment-13445241
 ] 

Mark Miller commented on SOLR-3750:
---

I suppose one way to kick the can is to make this configurable - which % of 
replicas to wait and see up before leader election, and how long to wait for 
that many to show up.

> On session expiration, we should explicitly wait some time before running the 
> leader sync process so that we are sure every node participates.
> --
>
> Key: SOLR-3750
> URL: https://issues.apache.org/jira/browse/SOLR-3750
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> We should wait until all the known nodes are part of the election, or X 
> amount of time if that does not happen (when a node or more does not come 
> back for whatever reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3762) NullPointerException when using grouping

[Jesse MacVicar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse MacVicar resolved SOLR-3762.
--

   Resolution: Fixed
Fix Version/s: 4.0

Looks like it's working good now, thanks Martijn!

> NullPointerException when using grouping
> 
>
> Key: SOLR-3762
> URL: https://issues.apache.org/jira/browse/SOLR-3762
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.0-BETA
>Reporter: Jesse MacVicar
> Fix For: 4.0
>
>
> Initial index is fine, seems to occur after additional documents have been 
> added/deleted. Simple index using grouping and group.facet. Full error posted 
> below.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3585) processing updates in multiple threads

[Dmitry Kan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445246#comment-13445246
 ] 

Dmitry Kan commented on SOLR-3585:
--

Summary:

1/2/4/8 threads

There was a gain for 2 threads, after that increasing amount of threads didn't 
matter for the indexing speed (again, can be too little data, too slow machine 
vs server)

URL:

http://localhost:8983/solr/update?commit=true&separator=%09&escape=\&update.chain=threads&backing.chain=logrun&stream.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&stream.contentType=text/csv;charset=utf-8

Intel(R) Core2 Duo CPU T6600 @ 2.20GHz
RAM: 4 GB
OS: Windows 7 64 bit

PC was moderately used during the indexing (Internet surfing mostly)

Solr started with:
java -Xmx512M -Xms512M -jar start.jar

Stats and Log extract:

---
one thread
---

565576 milliseconds (9.43 seconds)
size of data/index: 1.61 GB

30.08.2012 22:34:10 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update params={backing.chain=logrun&commi
t=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream
.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example
\data\book_edition.tsv&update.chain=threads} {add=[/m/0g9nk5p, /m/0g9rf0q, /m/0g
j6_r3, /m/0gj702y, /m/0gk99b7, /m/0g461_s, /m/0g4thbr, /m/0g4vp__, /m/0gkgw7x, /
m/0gb390f, ... (3401498 adds)]} 0 565576

---
two threads
---

400085 milliseconds (6.67 seconds)
size of data/index: 916MB

30.08.2012 22:09:16 org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1

30.08.2012 22:15:56 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update 
params={backing.chain=logrun&commit=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&update.chain=threads}
 {add=[/m/0g9nk5p, /m/0gj6_r3, /m/0gkgw7x, /m/0g9_qhd, /m/0g9_r1t, /m/0g9jxyt, 
/m/0g4wdtq, /m/0d0s9y1, /m/0d9pb_v, /m/0d0tfz7, ... (1838414 adds)]} 0 400085
30.08.2012 22:15:56 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update 
params={backing.chain=logrun&commit=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&update.chain=threads}
 {add=[/m/0g9rf0q, /m/0gj702y, /m/0gk99b7, /m/0g461_s, /m/0g4thbr, /m/0g4vp__, 
/m/0gb390f, /m/0gb34pf, /m/0h8fm59, /m/0g99vfk, ... (1563084 adds)]} 0 400085


---
four threads
---

423969 milliseconds (7.07 seconds)
size of data/index: 915 MB

30.08.2012 21:52:03 org.apache.solr.core.SolrDeletionPolicy updateCommits

INFO: [collection1] webapp=/solr path=/update 
params={backing.chain=logrun&commit=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&update.chain=threads}
 {add=[/m/0g9nk5p, /m/0dgjnsn, /m/0d0s539, /m/0d0t8b3, /m/0d9n2sg, /m/0d0s18j, 
/m/07n7lbm, /m/07n7mh6, /m/07n7mq0, /m/07n7n_d, ... (844367 adds)]} 0 r
30.08.2012 21:59:07 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] {add=[/m/0gj702y, /m/0gk99b7, /m/0gkgw7x, /m/0gb390f, 
/m/0g9_qhd, /m/0h2ymt3, /m/0g4wdtq, /m/0d0s9y1, /m/0d0tfz7, /m/0d0tdf1, ... 
(815450 adds)]} 0 423969
30.08.2012 21:59:07 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] {add=[/m/0g9rf0q, /m/0g461_s, /m/0g4thbr, /m/0g4vp__, 
/m/0gb34pf, /m/0h8fm59, /m/0g99vfk, /m/0g9_r1t, /m/0g9jxyt, /m/0ghc2b5, ... 
(836534 adds)]} 0 423969
30.08.2012 21:59:07 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update 
params={backing.chain=logrun&commit=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&update.chain=threads}
 {add=[/m/0gj6_r3, /m/0d0sfq_, /m/0d9mhx1, /m/07tc6lf, /m/07tc75v, /m/07tc7jq, 
/m/07tc8kz, /m/07tc8wr, /m/07tc_cn, /m/07tc_fl, ... (905147 adds)]} 0 423969

---
eight threads
---

431710 milliseconds (7.20 seconds)
size of data/index: 1.00 GB


30.08.2012 22:47:43 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update 
params={backing.chain=logrun&commit=true&stream.contentType=text/csv;charset%3Dutf-8&separator=%09&escape=\&stream
.file=d:\Projects\information_retrieval\solr\apache-solr-4.0.0-BETA\solr\example\data\book_edition.tsv&update.chain=threads}
 {add=[/

Re: Avoid losing data on ZK connection-loss/session-timeout

[Mark Miller]

> Ad 2) When the "behind" node has reconnected and become leader and the one
> with the latest updates does not come back live right away, isnt the new
> leader (which is behind) allowed to start handling update-requests. If yes,
> then it will be possible that both shards have documents/updates that the
> other one doesnt, and it is possible to come up with scenarios where there
> is no good algorithm for generating the "correct" merged union of the data
> in both shards. So what to do when the other shard (which used to have a
> later version than the current leader) comes live?
> 3) Believe there is nothing solid to do!
> How to avoid that? I was thinking about keeping the latest version for every
> slice in ZK, so that a "behind" shard will know if it has the latest version
> of a slice, and therefore if it is allowed to take the role as leader. Of
> course the writing of this "latest version" to ZK and the writing of the
> corresponding update in leaders transaction-log would have to be atomic
> (like the A in ACID) as much as possible. And it would be nice if writing of
> the update in replica transaction-log would also be atomic with the
> leader-writing and the ZK writing, in order to increase the chance that a
> replica is actually allowed to take over the leader role if the leader dies
> (or both dies and replica comes back first, and "old" leader comes back
> minutes later). But all that is just an idea on top of my head.
> Do you already have a solution implemented or a solution on the drawing
> board or how do you/we prevent such a problem? As far as I understand "the
> drill" during leader-election/recovery (whether its peer-sync or
> file-copy-replication) from the little code-reading I have done and from
> what you explain, there is not a current solution. But I might be wrong?
>

FWIW, I have added some logic so that we will wait to initiate leader
election until all the nodes are back or n amount of time goes by. I'd
like to make that configurable.

To back up though, no, I don't currently think any of this is a
problem currently anyway.

The way you would get updates on different nodes is that the leader
goes down in the middle of the update. If that happens, you don't know
if the update succeeded or not from the client. We don't return
success until every replica has the update. So the sync up stage
simply ensures that the shard stays consistent. Either an update that
you didn't know the success of will make it into the sync, or it
won't. Either way, since the client could not get a response, is fine
in my opinion. When you don't get a response, it's an open question
anyway.

Once we relax requiring the update gets to all replicas, then this may
be something I was more concerned about.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3750) On session expiration, we should explicitly wait some time before running the leader sync process so that we are sure every node participates.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445253#comment-13445253
 ] 

Mark Miller commented on SOLR-3750:
---

To add further - I don't think this is strictly necessary right now - due to 
the fact that we require updates to hit all replicas before returning success 
or failure. It will become a nice option once we let you be less strict with 
that though.

> On session expiration, we should explicitly wait some time before running the 
> leader sync process so that we are sure every node participates.
> --
>
> Key: SOLR-3750
> URL: https://issues.apache.org/jira/browse/SOLR-3750
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 4.0, 5.0
>
>
> We should wait until all the known nodes are part of the election, or X 
> amount of time if that does not happen (when a node or more does not come 
> back for whatever reason).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Dawid Weiss]

Steve is this script for truncating output applied to all console
output or just for e-mail messages? I ask because I look at:

http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/752/consoleText

and I see:

[...truncated too long message...]

We need those full outputs on the server to be able to look back and
see what happened. For e-mail it can be truncated but not on the
server. Either this, or we must preserve full output report files
(tests-report.txt).

Dawid




On Thu, Aug 30, 2012 at 5:46 PM, Dawid Weiss  wrote:
>
> I will investigate this later today. Thanks Steve.
>
> Sent from mobile phone.
>
> On Aug 30, 2012 5:18 PM, "Steven A Rowe"  wrote:
>>
>> There is a strange failure here in collecting test results (excerpted
>> below) - parsing of XML test output fails because there are missing closing
>> angle brackets on the last few elements in the file
>> :
>>
>> 114:> classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides"
>> name="testBefore" time="0.011"/>
>> 115:> classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides"
>> name="testAfter" time="0.011"/
>> 116:
>> 117:> 118:
>> 119:> 120:
>> 121: 
>>
>> Steve
>>
>> -Original Message-
>> From: Policeman Jenkins Server [mailto:jenk...@sd-datasolutions.de]
>> Sent: Thursday, August 30, 2012 10:57 AM
>> To: dev@lucene.apache.org; markrmil...@apache.org
>> Subject: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752
>> - Failure!
>>
>> [...]
>>
>> Recording test results
>> ERROR: Failed to archive test reports
>> hudson.util.IOException2: Failed to read
>> /var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
>> at hudson.tasks.junit.TestResult.parse(TestResult.java:244)
>> at hudson.tasks.junit.TestResult.parse(TestResult.java:163)
>> at hudson.tasks.junit.TestResult.parse(TestResult.java:140)
>> at hudson.tasks.junit.TestResult.(TestResult.java:116)
>> at
>> hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
>> at
>> hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
>> at hudson.FilePath.act(FilePath.java:842)
>> at hudson.FilePath.act(FilePath.java:824)
>> at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
>> at
>> hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:122)
>> at
>> hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:134)
>> at
>> hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
>> at
>> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
>> at
>> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)
>> at hudson.model.Build$BuildExecution.post2(Build.java:183)
>> at
>> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)
>> at hudson.model.Run.execute(Run.java:1527)
>> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
>> at
>> hudson.model.ResourceController.execute(ResourceController.java:88)
>> at hudson.model.Executor.run(Executor.java:236)
>> Caused by: org.dom4j.DocumentException: Error on line 115 of document
>> file:///var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/TEST-org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides.xml
>> : Element type "testcase" must be followed by either attribute
>> specifications, ">" or "/>". Nested exception: Element type "testcase" must
>> be followed by either attribute specifications, ">" or "/>".
>> at org.dom4j.io.SAXReader.read(SAXReader.java:482)
>> at org.dom4j.io.SAXReader.read(SAXReader.java:264)
>> at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:112)
>> at hudson.tasks.junit.TestResult.parse(TestResult.java:227)
>> ... 19 more
>> Caused by: org.xml.sax.SAXParseException: Element type "testcase" must be
>> followed by either attribute specifications, ">" or "/>".
>> at
>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
>> at
>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1427)
>> at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.seekCloseOfStartTag(XMLDocumentFragmentScannerImpl.java:1389)
>> at
>> com.sun.org.

Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Dawid Weiss]

> 114: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides" 
> name="testBefore" time="0.011"/>
> 115: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides" 
> name="testAfter" time="0.011"/
> 116:
> 117: 118:
> 119: 120:
> 121: 

Can't tell what the problem was here for now. The workspace is wiped
out (archive it if you see this error again, please).

The XML is clearly wrong but what's interesting is that these XMLs are
generated from an object model using simple-xml so there is no chance
for an unclosed tag unless there is something wrong with the library
itself. Seems unlikely -- we have been using it in production for a
long time without a glitch. I'll dig the next time we see this error
(if you happen to see it, please ZIP the contents of the workspace for
me).

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Steven A Rowe]

Will do. - Steve

-Original Message-
From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of Dawid 
Weiss
Sent: Thursday, August 30, 2012 4:34 PM
To: dev@lucene.apache.org
Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 
- Failure!

> 114: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides" 
> name="testBefore" time="0.011"/>
> 115: classname="org.apache.lucene.util.junitcompat.TestBeforeAfterOverrides" 
> name="testAfter" time="0.011"/
> 116:
> 117: 118:
> 119: 120:
> 121: 

Can't tell what the problem was here for now. The workspace is wiped
out (archive it if you see this error again, please).

The XML is clearly wrong but what's interesting is that these XMLs are
generated from an object model using simple-xml so there is no chance
for an unclosed tag unless there is something wrong with the library
itself. Seems unlikely -- we have been using it in production for a
long time without a glitch. I'll dig the next time we see this error
(if you happen to see it, please ZIP the contents of the workspace for
me).

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk7) - Build # 752 - Failure!

[Dawid Weiss]

> Will do. - Steve

Thanks!  How about the console logs -- are these truncated by that
script? Is there a way to separate what's remembered in jenkins's
history and what's e-mailed?

D.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org