[jira] [Resolved] (NUTCH-2828) Nightly builds fail with ivy error "Multiple artifacts of the module * are retrieved to the same file"
[
https://issues.apache.org/jira/browse/NUTCH-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel resolved NUTCH-2828.
Resolution: Fixed
Fixed by adding the ant target "clean-cache" (with fix for NUTCH-2829 applied)
in front of the other build targets ("clean, nightly, javadoc").
> Nightly builds fail with ivy error "Multiple artifacts of the module * are
> retrieved to the same file"
> --
>
> Key: NUTCH-2828
> URL: https://issues.apache.org/jira/browse/NUTCH-2828
> Project: Nutch
> Issue Type: Task
> Components: build
> Reporter: Sebastian Nagel
>Priority: Major
>
> Nightly builds fail since few weeks with an error while resolving ivy-managed
> dependencies:
> {noformat}
> /home/jenkins/jenkins-agent/workspace/Nutch/Nutch-trunk/src/plugin/build-plugin.xml:230:
> impossible to ivy retrieve: java.lang.RuntimeException: problem during
> retrieve of org.apache.nutch#lib-htmlunit: java.lang.RuntimeException:
> Multiple artifacts of the module com.squareup.okio#okio;1.14.0 are retrieved
> to the same file! Update the retrieve pattern to fix this error.
> {noformat}
> The error is not reproducible:
> - Github PR builds (NUTCH-2811) succeed
> - local builds on my dev machine also succeed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NUTCH-2828) Nightly builds fail with ivy error "Multiple artifacts of the module * are retrieved to the same file"
[
https://issues.apache.org/jira/browse/NUTCH-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel reassigned NUTCH-2828:
--
Assignee: Sebastian Nagel
> Nightly builds fail with ivy error "Multiple artifacts of the module * are
> retrieved to the same file"
> --
>
> Key: NUTCH-2828
> URL: https://issues.apache.org/jira/browse/NUTCH-2828
> Project: Nutch
> Issue Type: Task
> Components: build
>Reporter: Sebastian Nagel
>Assignee: Sebastian Nagel
>Priority: Major
>
> Nightly builds fail since few weeks with an error while resolving ivy-managed
> dependencies:
> {noformat}
> /home/jenkins/jenkins-agent/workspace/Nutch/Nutch-trunk/src/plugin/build-plugin.xml:230:
> impossible to ivy retrieve: java.lang.RuntimeException: problem during
> retrieve of org.apache.nutch#lib-htmlunit: java.lang.RuntimeException:
> Multiple artifacts of the module com.squareup.okio#okio;1.14.0 are retrieved
> to the same file! Update the retrieve pattern to fix this error.
> {noformat}
> The error is not reproducible:
> - Github PR builds (NUTCH-2811) succeed
> - local builds on my dev machine also succeed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NUTCH-2828) Nightly builds fail with ivy error "Multiple artifacts of the module * are retrieved to the same file"
Sebastian Nagel created NUTCH-2828:
--
Summary: Nightly builds fail with ivy error "Multiple artifacts of
the module * are retrieved to the same file"
Key: NUTCH-2828
URL: https://issues.apache.org/jira/browse/NUTCH-2828
Project: Nutch
Issue Type: Task
Components: build
Reporter: Sebastian Nagel
Nightly builds fail since few weeks with an error while resolving ivy-managed
dependencies:
{noformat}
/home/jenkins/jenkins-agent/workspace/Nutch/Nutch-trunk/src/plugin/build-plugin.xml:230:
impossible to ivy retrieve: java.lang.RuntimeException: problem during
retrieve of org.apache.nutch#lib-htmlunit: java.lang.RuntimeException: Multiple
artifacts of the module com.squareup.okio#okio;1.14.0 are retrieved to the same
file! Update the retrieve pattern to fix this error.
{noformat}
The error is not reproducible:
- Github PR builds (NUTCH-2811) succeed
- local builds on my dev machine also succeed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NUTCH-2771) Tests in nightly builds: speed up long runners
[
https://issues.apache.org/jira/browse/NUTCH-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel updated NUTCH-2771:
---
Fix Version/s: (was: 1.17)
1.18
> Tests in nightly builds: speed up long runners
> --
>
> Key: NUTCH-2771
> URL: https://issues.apache.org/jira/browse/NUTCH-2771
> Project: Nutch
> Issue Type: Improvement
> Components: build, test
>Affects Versions: 1.16
>Reporter: Sebastian Nagel
>Assignee: Sebastian Nagel
>Priority: Minor
> Fix For: 1.18
>
>
> The Nutch tests run by "ant test" or "ant nightly") take rather long to run.
> Although all tests are implemented as JUnit tests, some tests are more
> integration tests, eg. launching a Jetty web server and fetching documents
> from it. It's nice to have also higher level tests, and they are expected to
> long runner than a simple unit test. However, some of the test classes take
> really long to run (times taken from
> https://builds.apache.org/job/Nutch-trunk/3663/consoleText):
> {noformat}
> [junit] Running org.apache.nutch.segment.TestSegmentMergerCrawlDatums
> [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 133.898 sec
> [junit] Running org.apache.nutch.segment.TestSegmentMerger
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 101.026 sec
> [junit] Running org.apache.nutch.crawl.TestGenerator
> [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 46.03 sec
> [junit] Running org.apache.nutch.fetcher.TestFetcher
> [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 17.805 sec
> [junit] Running org.apache.nutch.urlfilter.fast.TestFastURLFilter
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 12.36 sec
> [junit] Running org.apache.nutch.parse.tika.TestPdfParser
> [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.974 sec
> [junit] Running org.apache.nutch.parse.tika.TestImageMetadata
> [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 9.113 sec
> [junit] Running org.apache.nutch.parse.feed.TestFeedParser
> [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6.369 sec
> [junit] Running org.apache.nutch.crawl.TestInjector
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6.15 sec
> {noformat}
> We could try to speed up at least some of these long-running tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NUTCH-2771) Tests in nightly builds: speed up long runners
[
https://issues.apache.org/jira/browse/NUTCH-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096297#comment-17096297
]
Sebastian Nagel commented on NUTCH-2771:
Moving to 1.18 for now. After a closer look: all these tests are useful. One
option could be to mark long-runners using [JUnit 5
tags|https://junit.org/junit5/docs/current/user-guide/#writing-tests-annotations]
which would allow to run them separately.
> Tests in nightly builds: speed up long runners
> --
>
> Key: NUTCH-2771
> URL: https://issues.apache.org/jira/browse/NUTCH-2771
> Project: Nutch
> Issue Type: Improvement
> Components: build, test
>Affects Versions: 1.16
>Reporter: Sebastian Nagel
>Assignee: Sebastian Nagel
>Priority: Minor
> Fix For: 1.17
>
>
> The Nutch tests run by "ant test" or "ant nightly") take rather long to run.
> Although all tests are implemented as JUnit tests, some tests are more
> integration tests, eg. launching a Jetty web server and fetching documents
> from it. It's nice to have also higher level tests, and they are expected to
> long runner than a simple unit test. However, some of the test classes take
> really long to run (times taken from
> https://builds.apache.org/job/Nutch-trunk/3663/consoleText):
> {noformat}
> [junit] Running org.apache.nutch.segment.TestSegmentMergerCrawlDatums
> [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 133.898 sec
> [junit] Running org.apache.nutch.segment.TestSegmentMerger
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 101.026 sec
> [junit] Running org.apache.nutch.crawl.TestGenerator
> [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 46.03 sec
> [junit] Running org.apache.nutch.fetcher.TestFetcher
> [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 17.805 sec
> [junit] Running org.apache.nutch.urlfilter.fast.TestFastURLFilter
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 12.36 sec
> [junit] Running org.apache.nutch.parse.tika.TestPdfParser
> [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 11.974 sec
> [junit] Running org.apache.nutch.parse.tika.TestImageMetadata
> [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 9.113 sec
> [junit] Running org.apache.nutch.parse.feed.TestFeedParser
> [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6.369 sec
> [junit] Running org.apache.nutch.crawl.TestInjector
> [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 6.15 sec
> {noformat}
> We could try to speed up at least some of these long-running tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NUTCH-2771) Tests in nightly builds: speed up long runners
Sebastian Nagel created NUTCH-2771:
--
Summary: Tests in nightly builds: speed up long runners
Key: NUTCH-2771
URL: https://issues.apache.org/jira/browse/NUTCH-2771
Project: Nutch
Issue Type: Improvement
Components: build, test
Affects Versions: 1.16
Reporter: Sebastian Nagel
Assignee: Sebastian Nagel
Fix For: 1.17
The Nutch tests run by "ant test" or "ant nightly") take rather long to run.
Although all tests are implemented as JUnit tests, some tests are more
integration tests, eg. launching a Jetty web server and fetching documents from
it. It's nice to have also higher level tests, and they are expected to long
runner than a simple unit test. However, some of the test classes take really
long to run (times taken from
https://builds.apache.org/job/Nutch-trunk/3663/consoleText):
{noformat}
[junit] Running org.apache.nutch.segment.TestSegmentMergerCrawlDatums
[junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
133.898 sec
[junit] Running org.apache.nutch.segment.TestSegmentMerger
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
101.026 sec
[junit] Running org.apache.nutch.crawl.TestGenerator
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
46.03 sec
[junit] Running org.apache.nutch.fetcher.TestFetcher
[junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
17.805 sec
[junit] Running org.apache.nutch.urlfilter.fast.TestFastURLFilter
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
12.36 sec
[junit] Running org.apache.nutch.parse.tika.TestPdfParser
[junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
11.974 sec
[junit] Running org.apache.nutch.parse.tika.TestImageMetadata
[junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
9.113 sec
[junit] Running org.apache.nutch.parse.feed.TestFeedParser
[junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
6.369 sec
[junit] Running org.apache.nutch.crawl.TestInjector
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
6.15 sec
{noformat}
We could try to speed up at least some of these long-running tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Nagel resolved NUTCH-1176. Resolution: Abandoned Outdated. > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Nightly builds
Great stuff, thanks Lewis On 8 January 2014 12:00, Lewis John Mcgibbney wrote: > Hi Folks, > > On Wed, Jan 8, 2014 at 4:06 AM, wrote: > >> I'm working on getting the Jenkins job configuration stable again. >> Something seems to have been reset or in not correct. >> I'll update here once we are back to stable builds. >> >> > Seems that there was an upgrade to the Jenkins servers we run the builds > on... which unfortunately resulted in this bug [0]. > > I made some tweaks to the job config and the good news is that builds are > back stable now. > > Ta > > [0] https://issues.jenkins-ci.org/browse/JENKINS-21250 > -- Open Source Solutions for Text Engineering http://digitalpebble.blogspot.com/ http://www.digitalpebble.com http://twitter.com/digitalpebble
Re: Nightly builds
Hi Folks, On Wed, Jan 8, 2014 at 4:06 AM, wrote: > I'm working on getting the Jenkins job configuration stable again. > Something seems to have been reset or in not correct. > I'll update here once we are back to stable builds. > > Seems that there was an upgrade to the Jenkins servers we run the builds on... which unfortunately resulted in this bug [0]. I made some tweaks to the job config and the good news is that builds are back stable now. Ta [0] https://issues.jenkins-ci.org/browse/JENKINS-21250
Nightly builds
Hi Folks, I'm working on getting the Jenkins job configuration stable again. Something seems to have been reset or in not correct. I'll update here once we are back to stable builds. Ta Lewis -- *Lewis*
Re: Failing Nightly Builds
Hi Tejas, I have no clue on this one I really don't. I've posted over on user@hadoop... hopefully something will offer itself forth to us. In the meantime, thanks for your interest. Best Lewis On Mon, Jan 7, 2013 at 9:32 PM, Tejas Patil wrote: > Hi Lewis, > > I feel that this issue might be surrounding "/etc/hosts" file. In [0], > Dennis Kubes suggested some change to the hosts file. (same thing was > mentioned in article [1]). In [2], the suggested to check if ssh works > using hostname and ip. > > [0] : > http://lucene.472066.n3.nabble.com/Nutch-Crawling-error-td612107.html > [1] : http://www.thegeekstuff.com/2012/02/hadoop-standalone-installation/ > [2] : http://www.mail-archive.com/user@cassandra.apache.org/msg16668.html > > Thanks, > Tejas Patil > > > On Mon, Jan 7, 2013 at 7:44 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com> wrote: > >> Hi Tejas, >> >> The Jenkins seems to have had a reboot (or something of this nature) >> around Christmas. I need to be honest and say that I don't know the source >> of the problem. >> Saying that, Hadoop (and other technologies) can also be a funny bugger >> sometimes when it comes to security, proxy, inet addresses. We've witnessed >> this a good few times over in Gora where HBase/Hadoop servers fail to >> initiate due to intermittent proxy settings/problems... I don't know enough >> about the code to provide a definitive answer. >> >> For the time being, I am happy to progress with integrating tests and >> minor tickets into Nutch, however I think we really ought to sort the >> source or this stack trace regardless of the fact it is on Jenkins. >> >> Maybe we should head over to hadoop general? >> >> I thought I would let this thread stew a while before pouncing on it >> again. >> >> Lewis >> >> On Mon, Jan 7, 2013 at 7:22 PM, Tejas Patil wrote: >> >>> Hi Lewis, >>> >>> These test cases pass on my machine (i guess on yours' too). Had it been >>> related to Hadoop API then tests must fail everywhere. What is different >>> about the setup where the nightly builds are executed ? >>> >>> Thanks, >>> Tejas Patil >>> >>> >>> On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney < >>> lewis.mcgibb...@gmail.com> wrote: >>> >>>> Hi All, >>>> >>>> An update to this issue then... >>>> >>>> The failing tests indicate an addition (security?) feature which makes >>>> the tests fail on Jenkins as it results in the following stack traces for >>>> the following tests >>>> >>>> TestCrawlDBMerger >>>> TestGenerator >>>> TestInjector >>>> TestLinkDBMerger >>>> TestFetcher >>>> TestSegmentMerger >>>> >>>> Same every time. >>>> Currently I am not sure how we can work around this, however I suspect >>>> that we need to use some other aspect of the Hadoop API in all calls to >>>> obtain socket addresses from servers. >>>> Any ideas? >>>> >>>> java.net.UnknownHostException: -s: -s >>>>at java.net.InetAddress.getLocalHost(InetAddress.java:1354) >>>>at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936) >>>>at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) >>>>at java.security.AccessController.doPrivileged(Native Method) >>>>at javax.security.auth.Subject.doAs(Subject.java:396) >>>>at >>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136) >>>>at >>>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) >>>>at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) >>>>at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323) >>>>at org.apache.nutch.crawl.Generator.generate(Generator.java:551) >>>>at org.apache.nutch.crawl.Generator.generate(Generator.java:465) >>>>at >>>> org.apache.nutch.crawl.TestGenerator.generateFetchlist(TestGenerator.java:313) >>>>at >>>> org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:259) >>>> >>>> >>>> >>>> -- >>>> *Lewis* >>>> >>> >>> >> >> >> -- >> *Lewis* >> > > -- *Lewis*
Re: Failing Nightly Builds
Hi Lewis, I feel that this issue might be surrounding "/etc/hosts" file. In [0], Dennis Kubes suggested some change to the hosts file. (same thing was mentioned in article [1]). In [2], the suggested to check if ssh works using hostname and ip. [0] : http://lucene.472066.n3.nabble.com/Nutch-Crawling-error-td612107.html [1] : http://www.thegeekstuff.com/2012/02/hadoop-standalone-installation/ [2] : http://www.mail-archive.com/user@cassandra.apache.org/msg16668.html Thanks, Tejas Patil On Mon, Jan 7, 2013 at 7:44 PM, Lewis John Mcgibbney < lewis.mcgibb...@gmail.com> wrote: > Hi Tejas, > > The Jenkins seems to have had a reboot (or something of this nature) > around Christmas. I need to be honest and say that I don't know the source > of the problem. > Saying that, Hadoop (and other technologies) can also be a funny bugger > sometimes when it comes to security, proxy, inet addresses. We've witnessed > this a good few times over in Gora where HBase/Hadoop servers fail to > initiate due to intermittent proxy settings/problems... I don't know enough > about the code to provide a definitive answer. > > For the time being, I am happy to progress with integrating tests and > minor tickets into Nutch, however I think we really ought to sort the > source or this stack trace regardless of the fact it is on Jenkins. > > Maybe we should head over to hadoop general? > > I thought I would let this thread stew a while before pouncing on it again. > > Lewis > > On Mon, Jan 7, 2013 at 7:22 PM, Tejas Patil wrote: > >> Hi Lewis, >> >> These test cases pass on my machine (i guess on yours' too). Had it been >> related to Hadoop API then tests must fail everywhere. What is different >> about the setup where the nightly builds are executed ? >> >> Thanks, >> Tejas Patil >> >> >> On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney < >> lewis.mcgibb...@gmail.com> wrote: >> >>> Hi All, >>> >>> An update to this issue then... >>> >>> The failing tests indicate an addition (security?) feature which makes >>> the tests fail on Jenkins as it results in the following stack traces for >>> the following tests >>> >>> TestCrawlDBMerger >>> TestGenerator >>> TestInjector >>> TestLinkDBMerger >>> TestFetcher >>> TestSegmentMerger >>> >>> Same every time. >>> Currently I am not sure how we can work around this, however I suspect >>> that we need to use some other aspect of the Hadoop API in all calls to >>> obtain socket addresses from servers. >>> Any ideas? >>> >>> java.net.UnknownHostException: -s: -s >>> at java.net.InetAddress.getLocalHost(InetAddress.java:1354) >>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936) >>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:396) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136) >>> at >>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) >>> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) >>> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323) >>> at org.apache.nutch.crawl.Generator.generate(Generator.java:551) >>> at org.apache.nutch.crawl.Generator.generate(Generator.java:465) >>> at >>> org.apache.nutch.crawl.TestGenerator.generateFetchlist(TestGenerator.java:313) >>> at >>> org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:259) >>> >>> >>> >>> -- >>> *Lewis* >>> >> >> > > > -- > *Lewis* >
Re: Failing Nightly Builds
Hi Tejas, The Jenkins seems to have had a reboot (or something of this nature) around Christmas. I need to be honest and say that I don't know the source of the problem. Saying that, Hadoop (and other technologies) can also be a funny bugger sometimes when it comes to security, proxy, inet addresses. We've witnessed this a good few times over in Gora where HBase/Hadoop servers fail to initiate due to intermittent proxy settings/problems... I don't know enough about the code to provide a definitive answer. For the time being, I am happy to progress with integrating tests and minor tickets into Nutch, however I think we really ought to sort the source or this stack trace regardless of the fact it is on Jenkins. Maybe we should head over to hadoop general? I thought I would let this thread stew a while before pouncing on it again. Lewis On Mon, Jan 7, 2013 at 7:22 PM, Tejas Patil wrote: > Hi Lewis, > > These test cases pass on my machine (i guess on yours' too). Had it been > related to Hadoop API then tests must fail everywhere. What is different > about the setup where the nightly builds are executed ? > > Thanks, > Tejas Patil > > > On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com> wrote: > >> Hi All, >> >> An update to this issue then... >> >> The failing tests indicate an addition (security?) feature which makes >> the tests fail on Jenkins as it results in the following stack traces for >> the following tests >> >> TestCrawlDBMerger >> TestGenerator >> TestInjector >> TestLinkDBMerger >> TestFetcher >> TestSegmentMerger >> >> Same every time. >> Currently I am not sure how we can work around this, however I suspect >> that we need to use some other aspect of the Hadoop API in all calls to >> obtain socket addresses from servers. >> Any ideas? >> >> java.net.UnknownHostException: -s: -s >> at java.net.InetAddress.getLocalHost(InetAddress.java:1354) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136) >> at >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) >> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) >> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323) >> at org.apache.nutch.crawl.Generator.generate(Generator.java:551) >> at org.apache.nutch.crawl.Generator.generate(Generator.java:465) >> at >> org.apache.nutch.crawl.TestGenerator.generateFetchlist(TestGenerator.java:313) >> at >> org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:259) >> >> >> >> -- >> *Lewis* >> > > -- *Lewis*
Re: Failing Nightly Builds
Hi Lewis, These test cases pass on my machine (i guess on yours' too). Had it been related to Hadoop API then tests must fail everywhere. What is different about the setup where the nightly builds are executed ? Thanks, Tejas Patil On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney < lewis.mcgibb...@gmail.com> wrote: > Hi All, > > An update to this issue then... > > The failing tests indicate an addition (security?) feature which makes the > tests fail on Jenkins as it results in the following stack traces for the > following tests > > TestCrawlDBMerger > TestGenerator > TestInjector > TestLinkDBMerger > TestFetcher > TestSegmentMerger > > Same every time. > Currently I am not sure how we can work around this, however I suspect > that we need to use some other aspect of the Hadoop API in all calls to > obtain socket addresses from servers. > Any ideas? > > java.net.UnknownHostException: -s: -s > at java.net.InetAddress.getLocalHost(InetAddress.java:1354) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323) > at org.apache.nutch.crawl.Generator.generate(Generator.java:551) > at org.apache.nutch.crawl.Generator.generate(Generator.java:465) > at > org.apache.nutch.crawl.TestGenerator.generateFetchlist(TestGenerator.java:313) > at > org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:259) > > > > -- > *Lewis* >
[jira] [Updated] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Markus Jelsma updated NUTCH-1176: - Fix Version/s: (was: 1.5) (was: nutchgora) 1.6 20120304-push-1.6 > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > Fix For: 1.6 > > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186307#comment-13186307 ] Hudson commented on NUTCH-1176: --- Integrated in nutch-trunk-maven #110 (See [https://builds.apache.org/job/nutch-trunk-maven/110/]) NUTCH-1176 commit lewismc : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=&revision=1231532 Files : * /nutch/trunk/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/crawl/FetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/parse/ParseResult.java > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > Fix For: nutchgora, 1.5 > > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186298#comment-13186298 ] Hudson commented on NUTCH-1176: --- Integrated in Nutch-trunk #1728 (See [https://builds.apache.org/job/Nutch-trunk/1728/]) NUTCH-1176 commit lewismc : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=.&revision=1231532 Files : * /nutch/trunk/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/crawl/FetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/parse/ParseResult.java > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > Fix For: nutchgora, 1.5 > > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186255#comment-13186255 ] Hudson commented on NUTCH-1176: --- Integrated in nutch-trunk-maven #108 (See [https://builds.apache.org/job/nutch-trunk-maven/108/]) commit to try and resolve NUTCH-1176, I expect this not to work 1st time, N.B. This doesn't change or even touch syntax of code. lewismc : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=&revision=1231517 Files : * /nutch/trunk/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/crawl/FetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrDeleteDuplicates.java * /nutch/trunk/src/java/org/apache/nutch/tools/arc/ArcRecordReader.java * /nutch/trunk/src/java/org/apache/nutch/util/MimeUtil.java * /nutch/trunk/src/java/org/apache/nutch/util/NodeWalker.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/DomainSuffix.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/DomainSuffixes.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/TopLevelDomain.java * /nutch/trunk/src/plugin/lib-regex-filter/src/java/org/apache/nutch/urlfilter/api/RegexURLFilterBase.java > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > Fix For: nutchgora, 1.5 > > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1176) Fix all javadoc warnings from nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186244#comment-13186244 ] Hudson commented on NUTCH-1176: --- Integrated in Nutch-trunk #1726 (See [https://builds.apache.org/job/Nutch-trunk/1726/]) commit to try and resolve NUTCH-1176, I expect this not to work 1st time, N.B. This doesn't change or even touch syntax of code. lewismc : http://svn.apache.org/viewvc/nutch/trunk/viewvc/?view=rev&root=.&revision=1231517 Files : * /nutch/trunk/src/java/org/apache/nutch/crawl/AbstractFetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/crawl/FetchSchedule.java * /nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrDeleteDuplicates.java * /nutch/trunk/src/java/org/apache/nutch/tools/arc/ArcRecordReader.java * /nutch/trunk/src/java/org/apache/nutch/util/MimeUtil.java * /nutch/trunk/src/java/org/apache/nutch/util/NodeWalker.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/DomainSuffix.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/DomainSuffixes.java * /nutch/trunk/src/java/org/apache/nutch/util/domain/TopLevelDomain.java * /nutch/trunk/src/plugin/lib-regex-filter/src/java/org/apache/nutch/urlfilter/api/RegexURLFilterBase.java > Fix all javadoc warnings from nightly builds > > > Key: NUTCH-1176 > URL: https://issues.apache.org/jira/browse/NUTCH-1176 > Project: Nutch > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4, nutchgora >Reporter: Lewis John McGibbney >Priority: Minor > Fix For: nutchgora, 1.5 > > > The warnings can clearly be seen from the javadoc target (near bottom) of any > successful nightly build. An example is provided below. > https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (NUTCH-1176) Fix all javadoc warnings from nightly builds
Fix all javadoc warnings from nightly builds Key: NUTCH-1176 URL: https://issues.apache.org/jira/browse/NUTCH-1176 Project: Nutch Issue Type: Improvement Components: documentation Affects Versions: 1.4, nutchgora Reporter: Lewis John McGibbney Priority: Minor Fix For: nutchgora, 1.5 The warnings can clearly be seen from the javadoc target (near bottom) of any successful nightly build. An example is provided below. https://builds.apache.org/job/nutch-trunk/1638/console -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (NUTCH-823) Download page should not have pointer to nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebb closed NUTCH-823. -- Thanks > Download page should not have pointer to nightly builds > --- > > Key: NUTCH-823 > URL: https://issues.apache.org/jira/browse/NUTCH-823 > Project: Nutch > Issue Type: Bug >Reporter: Sebb >Assignee: Chris A. Mattmann >Priority: Minor > Fix For: 1.2, 2.0 > > > The download page > http://www.apache.org/dist/lucene/nutch/ > has a pointer to nightly builds. These are not supposed to be advertised to > the general public, see: > http://www.apache.org/dev/release.html#what > "Do not include any links on the project website that might encourage > non-developers to download and use nightly builds, snapshots, release > candidates, or any other similar package. " -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (NUTCH-823) Download page should not have pointer to nightly builds
[
https://issues.apache.org/jira/browse/NUTCH-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris A. Mattmann resolved NUTCH-823.
-
Assignee: Chris A. Mattmann
Resolution: Fixed
Hi Sebb:
OK, took care of it this time, looks like for good. Here are the contents of
the Nutch HEADER.html file:
{noformat}
# more HEADER.html
http://nutch.apache.org/";>Nutch Releases
Please make sure you're downloading from http://www.apache.org/dyn/closer.cgi/nutch/";>a nearby
mirror site, not from www.apache.org.
{noformat}
The mirrors should pick up the updated HEADER.html shortly...
Cheers,
Chris
> Download page should not have pointer to nightly builds
> ---
>
> Key: NUTCH-823
> URL: https://issues.apache.org/jira/browse/NUTCH-823
> Project: Nutch
> Issue Type: Bug
>Reporter: Sebb
>Assignee: Chris A. Mattmann
>Priority: Minor
> Fix For: 1.2, 2.0
>
>
> The download page
> http://www.apache.org/dist/lucene/nutch/
> has a pointer to nightly builds. These are not supposed to be advertised to
> the general public, see:
> http://www.apache.org/dev/release.html#what
> "Do not include any links on the project website that might encourage
> non-developers to download and use nightly builds, snapshots, release
> candidates, or any other similar package. "
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (NUTCH-823) Download page should not have pointer to nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebb reopened NUTCH-823: The old directory under lucene is now empty, however: http://www.apache.org/dist/nutch/ still contains a link to nightly builds, so the problem has not been resolved - just moved. > Download page should not have pointer to nightly builds > --- > > Key: NUTCH-823 > URL: https://issues.apache.org/jira/browse/NUTCH-823 > Project: Nutch > Issue Type: Bug >Reporter: Sebb >Priority: Minor > Fix For: 1.2, 2.0 > > > The download page > http://www.apache.org/dist/lucene/nutch/ > has a pointer to nightly builds. These are not supposed to be advertised to > the general public, see: > http://www.apache.org/dev/release.html#what > "Do not include any links on the project website that might encourage > non-developers to download and use nightly builds, snapshots, release > candidates, or any other similar package. " -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (NUTCH-823) Download page should not have pointer to nightly builds
[ https://issues.apache.org/jira/browse/NUTCH-823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris A. Mattmann resolved NUTCH-823. - Fix Version/s: 1.2 2.0 Resolution: Fixed - since the site was updated, this issue is fixed. > Download page should not have pointer to nightly builds > --- > > Key: NUTCH-823 > URL: https://issues.apache.org/jira/browse/NUTCH-823 > Project: Nutch > Issue Type: Bug >Reporter: Sebb >Priority: Minor > Fix For: 1.2, 2.0 > > > The download page > http://www.apache.org/dist/lucene/nutch/ > has a pointer to nightly builds. These are not supposed to be advertised to > the general public, see: > http://www.apache.org/dev/release.html#what > "Do not include any links on the project website that might encourage > non-developers to download and use nightly builds, snapshots, release > candidates, or any other similar package. " -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (NUTCH-823) Download page should not have pointer to nightly builds
Download page should not have pointer to nightly builds --- Key: NUTCH-823 URL: https://issues.apache.org/jira/browse/NUTCH-823 Project: Nutch Issue Type: Bug Reporter: Sebb Priority: Minor The download page http://www.apache.org/dist/lucene/nutch/ has a pointer to nightly builds. These are not supposed to be advertised to the general public, see: http://www.apache.org/dev/release.html#what "Do not include any links on the project website that might encourage non-developers to download and use nightly builds, snapshots, release candidates, or any other similar package. " -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
