[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785948#comment-16785948 ] Kevin Risden commented on SOLR-9515: Thanks for pointing those out. I saw them on the report and saw a few that were 7.x related. I'll see if I can figure out what is going on with them. I have looked into a bunch of the HDFS related tests on master/8.x. Those 2 were not on my list. I'm going to create another Jira to track these down. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785896#comment-16785896 ] Hoss Man commented on SOLR-9515: Kevin: StressHdfsTest and HdfsUnloadDistributedZkTest have both should an uptick in their jenkins failure rates since this change -- I thought i remembered you saying you were investigating hadoop related test failures since the upgrade, but i haven't seen any recent commits or new jiras related to these tests so i thought i'd point it out explicitly > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763638#comment-16763638 ] Kevin Risden commented on SOLR-9515: I was never able to figure out why locally "ant run-maven-build -DskipTests=true" fails but on my local jenkins I was finally able to get it to work. [https://builds.apache.org/job/Lucene-Solr-Maven-master/2484/] The job failed due to forbiddenapi pattern not being updated in pom.xml.template. Fixing with commit shortly. Confirmed that Maven build is now successful with this change. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763641#comment-16763641 ] ASF subversion and git services commented on SOLR-9515: --- Commit 4038f14ac1594f14cc1c83bcfc28e855766f37a4 in lucene-solr's branch refs/heads/branch_8_0 from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4038f14 ] SOLR-9515 - Add maven forbiddenapis exclude for copied Hadoop code Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763640#comment-16763640 ] ASF subversion and git services commented on SOLR-9515: --- Commit 092b22faa3b3edf8e4a96a63c8eef75c83c4305f in lucene-solr's branch refs/heads/branch_8x from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=092b22f ] SOLR-9515 - Add maven forbiddenapis exclude for copied Hadoop code Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763639#comment-16763639 ] ASF subversion and git services commented on SOLR-9515: --- Commit 796fbaef766671c46b55263d750c5824fb13e8fb in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=796fbae ] SOLR-9515 - Add maven forbiddenapis exclude for copied Hadoop code Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, > SOLR-9515-forbiddenapis-maven.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762852#comment-16762852 ] ASF subversion and git services commented on SOLR-9515: --- Commit bef1658e68ad96cb2df5b4bafc8607c21ab4530d in lucene-solr's branch refs/heads/branch_8_0 from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=bef1658 ] SOLR-9515 - Remove pom dependency kerby-kerb Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762848#comment-16762848 ] ASF subversion and git services commented on SOLR-9515: --- Commit 475f2fd36b374f30f8f8e3084e7103ccf5416cbe in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=475f2fd ] SOLR-9515 - Remove pom dependency kerby-kerb Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762850#comment-16762850 ] ASF subversion and git services commented on SOLR-9515: --- Commit fd061bddf2ddd02b557c7e9a40cb74f543634a65 in lucene-solr's branch refs/heads/branch_8x from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=fd061bd ] SOLR-9515 - Remove pom dependency kerby-kerb Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515-fix_pom.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762835#comment-16762835
]
Kevin Risden commented on SOLR-9515:
Thanks [~steve_rowe] - I am running "ant run-maven-build -DskipTests=true" and
after fixing the kerby-kerb dependency, I am getting the following error
locally.
{code:java}
[mvn] [INFO] Failed to resolve artifact.
[mvn]
[mvn] Unable to get dependency information: Unable to read the metadata file
for artifact 'com.healthmarketscience.jackcess:jackcess:jar': Cannot find
parent: com.healthmarketscience:openhms-parent for project:
com.healthmarketscience.jackcess:jackcess:jar:2.1.12 for project
com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn] com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn]
[mvn] from the specified remote repositories:
[mvn] maven-restlet (http://maven.restlet.org),
[mvn] central (http://repo1.maven.org/maven2),
[mvn] releases.cloudera.com
(https://repository.cloudera.com/artifactory/libs-release),
[mvn] apache.snapshots (http://repository.apache.org/snapshots)
[mvn]
[mvn] Path to dependency:
[mvn] 1) org.apache.solr:solr-dataimporthandler-extras:jar:9.0.0-SNAPSHOT
[mvn]
[mvn]
[mvn]
[mvn] [INFO]
[mvn] [INFO] Error for project: Apache Solr Content Extraction Library (during
install)
[mvn] [INFO]
[mvn] [INFO] Failed to resolve artifact.
[mvn]
[mvn] Unable to get dependency information: Unable to read the metadata file
for artifact 'com.healthmarketscience.jackcess:jackcess:jar': Cannot find
parent: com.healthmarketscience:openhms-parent for project:
com.healthmarketscience.jackcess:jackcess:jar:2.1.12 for project
com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn] com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn]
[mvn] from the specified remote repositories:
[mvn] maven-restlet (http://maven.restlet.org),
[mvn] central (http://repo1.maven.org/maven2),
[mvn] releases.cloudera.com
(https://repository.cloudera.com/artifactory/libs-release),
[mvn] apache.snapshots (http://repository.apache.org/snapshots)
[mvn]
[mvn] Path to dependency:
[mvn] 1) org.apache.solr:solr-cell:jar:9.0.0-SNAPSHOT
[mvn]
[mvn]
[mvn]
[mvn] [INFO]
[mvn] [INFO] Error for project: Apache Solr Language Identifier (during install)
[mvn] [INFO]
[mvn] [INFO] Failed to resolve artifact.
[mvn]
[mvn] Unable to get dependency information: Unable to read the metadata file
for artifact 'com.healthmarketscience.jackcess:jackcess:jar': Cannot find
parent: com.healthmarketscience:openhms-parent for project:
com.healthmarketscience.jackcess:jackcess:jar:2.1.12 for project
com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn] com.healthmarketscience.jackcess:jackcess:jar:2.1.12
[mvn]
[mvn] from the specified remote repositories:
[mvn] maven-restlet (http://maven.restlet.org),
[mvn] central (http://repo1.maven.org/maven2),
[mvn] releases.cloudera.com
(https://repository.cloudera.com/artifactory/libs-release),
[mvn] apache.snapshots (http://repository.apache.org/snapshots)
[mvn]
[mvn] Path to dependency:
[mvn] 1) org.apache.solr:solr-langid:jar:9.0.0-SNAPSHOT
[mvn]
[mvn]
[mvn]
[mvn] [INFO]
[mvn] [INFO] For more information, run Maven with the -e switch
[mvn] [INFO]
[mvn] [INFO] BUILD ERRORS
[mvn] [INFO]
[mvn] [INFO] Total time: 2 minutes 33 seconds
[mvn] [INFO] Finished at: Thu Feb 07 11:30:48 EST 2019
[mvn] [INFO] Final Memory: 332M/1769M
[mvn] [INFO]
BUILD FAILED
: Java returned: 1
{code}
This isn't related to the Hadoop 3 upgrade since that dependency wasn't touched.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762779#comment-16762779
]
Steve Rowe commented on SOLR-9515:
--
Maven build usage instructions are at {{dev-tools/maven/README. maven}}. FYI
currently the Maven Jenkins jobs don't run tests under Maven because nobody
maintains that capability - the Maven test runner (surefire) is different from
RandomizedRunner, the Ant one (see LUCENE-4045); the Jenkins jobs just build
the Maven artifacts and populate the Apache Maven snapshot repo.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762660#comment-16762660 ] Kevin Risden commented on SOLR-9515: [~steve_rowe] - Thanks I saw the same build fail and was planning to follow up today. For next time, is there a way to check the Maven part locally? > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, 8.x, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762377#comment-16762377
]
Steve Rowe commented on SOLR-9515:
--
The Maven build has been failing since this was committed because the
kerby-kerb test dependency can't be found, e.g. from
[https://builds.apache.org/job/Lucene-Solr-Maven-master/2483]:
{noformat}
[mvn] [INFO] Apache Solr Core tests
FAILED [1.525s]
[...]
[mvn] [INFO]
[mvn] [INFO] Error for project: Apache Solr Core tests (during install)
[mvn] [INFO]
[mvn] [INFO] Failed to resolve artifact.
[mvn]
[mvn] Missing:
[mvn] --
[mvn] 1) org.apache.kerby:kerby-kerb:jar:1.0.1
[mvn]
[mvn] Try downloading the file manually from the project website.
[mvn]
[mvn] Then, install it using the command:
[mvn] mvn install:install-file -DgroupId=org.apache.kerby
-DartifactId=kerby-kerb -Dversion=1.0.1 -Dpackaging=jar -Dfile=/path/to/file
[mvn]
[mvn] Alternatively, if you host your own repository you can deploy the
file there:
[mvn] mvn deploy:deploy-file -DgroupId=org.apache.kerby
-DartifactId=kerby-kerb -Dversion=1.0.1 -Dpackaging=jar -Dfile=/path/to/file
-Durl=[url] -DrepositoryId=[id]
[mvn]
[mvn] Path to dependency:
[mvn] 1) org.apache.solr:solr-core-tests:jar:9.0.0-SNAPSHOT
[mvn] 2) org.apache.kerby:kerby-kerb:jar:1.0.1
{noformat}
The reason: {{kerby-kerb}} is an aggregation pom, not a jar. See
https://search.maven.org/artifact/org.apache.kerby/kerby-kerb/1.0.1/pom : note
{{pom}}.
It can be safely removed from {{solr/core/ivy.xml}} (and from
{{lucene/ivy-versions.properties}}, since it's only used in the one place).
When I remove it, {{ant test}} succeeds for me under {{solr/core/}}.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, 8.x, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759033#comment-16759033 ] Kevin Risden commented on SOLR-9515: Thanks to [~markrmil...@gmail.com], [~thetaphi], [~dsmiley], [~gerlowskija] and anyone I might have missed for the work and reviews on this! > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0), 8.x > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759032#comment-16759032 ] Kevin Risden commented on SOLR-9515: Committed to 8x and 8.0. Resolving since this work is done currently. Will keep an eye on Jenkins over the next few days to make sure there are no issues. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0), 8.x > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759031#comment-16759031 ] ASF subversion and git services commented on SOLR-9515: --- Commit 6dd22af2f2a02ec8c81d78ec1a39c89fcad0f378 in lucene-solr's branch refs/heads/branch_8_0 from Mark Robert Miller [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6dd22af ] SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden) Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0), 8.x > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759026#comment-16759026 ] ASF subversion and git services commented on SOLR-9515: --- Commit f20d2d01544c631ef55c529f4d5a07d6f7e645ed in lucene-solr's branch refs/heads/branch_8x from Mark Robert Miller [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f20d2d0 ] SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden) Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0), 8.x > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759021#comment-16759021 ] Kevin Risden commented on SOLR-9515: The following builds have also now passed with no HDFS failures: * Windows, Mac, Solaris, and two more linux builds - jenkins.thetaphi.de. * Linux master and Linux nightly master - [https://builds.apache.org|https://builds.apache.org/] I am going to merge the change to 8.x and 8.0 to get some test runs over the weekend. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758634#comment-16758634 ] ASF subversion and git services commented on SOLR-9515: --- Commit 4a3ddc94d8880f7c1f76dedbe7d892b0542bb6ee in lucene-solr's branch refs/heads/master from Mark Robert Miller [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4a3ddc9 ] SOLR-9515: Update to Hadoop 3 (Mark Miller, Kevin Risden) Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 7h 10m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758631#comment-16758631 ] Kevin Risden commented on SOLR-9515: Squashed commits back down to a single commit to make it easier to commit. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 6h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758429#comment-16758429 ] Kevin Risden commented on SOLR-9515: Updated patch with BlockSlicePool workaround. First run of tests locally with JDK8 and JDK11 look promising. I have a few machines running the tests to see if anything shakes out. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 6h 40m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758279#comment-16758279 ] Kevin Risden commented on SOLR-9515: Created HDFS-14251 with the Hadoop project. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758277#comment-16758277 ] Kevin Risden commented on SOLR-9515: Thanks [~markrmil...@gmail.com] and [~thetaphi]. I'll open a ticket with Hadoop. I started to patch the Hadoop code to pass in a custom factory. I can improve that to pass in custom thread names as well. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758049#comment-16758049 ] Uwe Schindler commented on SOLR-9515: - IMHO, as we can now provide our own thread factory, we may use the same approach like in the TestFramework for normal executors, where we have a thread factory that names our threads in a better way. This would allow to possibly remove the special cases in the thread leak detector. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758034#comment-16758034 ] Uwe Schindler commented on SOLR-9515: - I agree with Mark Miller. BTW, the commonPool and the ForkJoin pools are made for cpu-intensive calculation, so in general they won't need any special permissions. What Hadoop is doing here is very untypical, because it misuses the fork-join-pool for I/O stuff which generally just waits for I/O devices to finish, so it's not a cpu-intensive job. This especially applies for the common pool (everybody in Java says: Whenever you do something with common-pool like stream.parallel() never ever do I/O inside. The change in Java 9 now unfortunately changes this to all fork join pools. I think we should file a bug to change hadoop to use their own thread factory or use another pool. bq. Definitely didn't work. From the ForkJoinPool javadocs The sentence in the docs was only meant for the "commonPool", so when you change that to 0 it does not create any threads at all, so stream().parallel() is not doing anything. Hadoop is using its own ForkJoin pool that breaks. I am fine with the same approach for tests that we did for the Jetty server (it's a test only class here): Fork it and fix it until Hadoop fixed their bug. We'd really open a bug report. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757960#comment-16757960 ] Mark Miller commented on SOLR-9515: --- You should file a Hadoop Jira and see what they say. I was using a bunch of ForkJoinPool in the large test cleanup I did and ran into issues like this so replaced it with a normal executor. That thing is too special and more trouble than it's worth by far. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757883#comment-16757883
]
Lucene/Solr QA commented on SOLR-9515:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
7s{color} | {color:green} tools in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 40m
33s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-9515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957138/SOLR-9515.patch |
| Optional Tests | checklicenses validatesourcepatterns ratsources compile
javac unit checkforbiddenapis |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / e4f202c |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_191 |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/284/testReport/ |
| modules | C: lucene lucene/tools solr solr/core solr/test-framework U: . |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/284/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757876#comment-16757876 ] Kevin Risden commented on SOLR-9515: So I see three ways forward short term - none of which I really like. # Disable Hadoop tests on JDK9+ ## Part of this Hadoop 3 upgrade was to make it possible to run on JDK9+ potentially with Hadoop support. ## There are other Hadoop tests currently disabled with JDK9+ due to existing issues. # Copy and patch BlockPoolSlice to change how the ForkJoinPool is created ## BlockPoolSlice looks to be the only place where ForkJoinPool will cause issues for the tests. I am currently validating this. ## We would end up with 2 classes copied to make the Hadoop integration tests work. # Don't upgrade to Hadoop 3 until Hadoop is more SecurityManager friendly. Long term it would be good to fix Hadoop to make it more SecurityManager friendly. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757855#comment-16757855 ] Kevin Risden commented on SOLR-9515: I found why this breaks on JDK9+ but not on JDK8. The DefaultForkJoinWorkerThreadFactory implementation changed to have no default permissions. JDK8 - [http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/concurrent/ForkJoinPool.java#l587] JDK11 - http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#l705 > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757839#comment-16757839 ] Kevin Risden commented on SOLR-9515: ForkJoinPool was added in HDFS-13768 > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757838#comment-16757838 ] Kevin Risden commented on SOLR-9515: Reverted the hadoop 3 commit since it is broken with JDK9+ and the security manager. Hadoop uses ForkJoinPool which by default creates threads that have no permissions. There is no easy way to work around the ForkJoinPool currently. Thinking about ways to move forward. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757834#comment-16757834 ] ASF subversion and git services commented on SOLR-9515: --- Commit e4f202c1e30f7c7209f978d7733922245c33ab71 in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e4f202c ] Revert "SOLR-9515: Update to Hadoop 3" This reverts commit 6bb24673f422a4e4267bc22361bc9258809d5f60. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757807#comment-16757807
]
Kevin Risden commented on SOLR-9515:
Well setting that key to 0 results in the following.
{code:java}
com.carrotsearch.randomizedtesting.UncaughtExceptionError:
Captured an uncaught exception in thread: Thread[id=226, name=Thread-111,
state=RUNNABLE, group=TGRP-HdfsRecoverLeaseTest]
Caused by: java.lang.IllegalArgumentException
at __randomizedtesting.SeedInfo.seed([B3EBC148FC827CD8]:0)
at
java.base/java.util.concurrent.ForkJoinPool.init(ForkJoinPool.java:2295)
at
java.base/java.util.concurrent.ForkJoinPool.init(ForkJoinPool.java:2165)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.initializeAddReplicaPool(BlockPoolSlice.java:213)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.init(BlockPoolSlice.java:188)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addBlockPool(FsVolumeImpl.java:1041)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.addBlockPool(FsVolumeImpl.java:1033)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeList$2.run(FsVolumeList.java:412)
{code}
Definitely didn't work. From the ForkJoinPool javadocs:
@throws IllegalArgumentException if parallelism less than or equal to zero, or
greater than implementation limit
So yea not sure how to work around this.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757804#comment-16757804 ] Kevin Risden commented on SOLR-9515: Yea not sure how to work around it either. The pool is made explicitly by BlockPoolSlice so none of the system properties take affect. I don't understand how this works on JDK8 though. I'll have to take a break and look at it again later tonight or tomorrow. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757805#comment-16757805 ] Kevin Risden commented on SOLR-9515: Ah our comments overlapped let me try that. That is pretty easy to set. Thanks for looking! > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757800#comment-16757800 ] Uwe Schindler commented on SOLR-9515: - This one in test setup: DFSConfigKeys.DFS_DATANODE_VOLUMES_REPLICA_ADD_THREADPOOL_SIZE_KEY to 0 > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757793#comment-16757793 ] Uwe Schindler commented on SOLR-9515: - You found the issue. Not sure how to work around that. Maybe set parallelism to 0 on test runner properties. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757797#comment-16757797 ] Uwe Schindler commented on SOLR-9515: - Or set the parallelism to 0 in the DFSConfig in test setup. That should be the easiest. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757791#comment-16757791
]
Kevin Risden commented on SOLR-9515:
Well looks like I found something that is promising. I don't know why this
would work on JDK8 though.
[https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html]
{code:java}
If a SecurityManager is present and no factory is specified, then the default
pool uses a factory supplying threads that have no Permissions enabled. The
system class loader is used to load these classes. Upon any error in
establishing these settings, default parameters are used. It is possible to
disable or limit the use of threads in the common pool by setting the
parallelism property to zero, and/or using a factory that may return null.
However doing so may cause unjoined tasks to never be executed.
{code}
[https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ForkJoinPool.html]
{code:java}
If no thread factory is supplied via a system property, then the common pool
uses a factory that uses the system class loader as the thread context class
loader. In addition, if a SecurityManager is present, then the common pool uses
a factory supplying threads that have no Permissions enabled. Upon any error in
establishing these settings, default parameters are used. It is possible to
disable or limit the use of threads in the common pool by setting the
parallelism property to zero, and/or using a factory that may return null.
However doing so may cause unjoined tasks to never be executed.{code}
From
https://github.com/apache/hadoop/blob/branch-3.2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java#L213
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757778#comment-16757778 ] Uwe Schindler commented on SOLR-9515: - Interestingly, the Solr test policy (in contrast to Lucene's) allows read access everywhere: first line with "<<>>" I am currently not know what's going on. I'd like to help, but I am very busy because I have to fly to FOSDEM conference tomorrow and before that have a phone conference, so I have to sleep. The only thing you can do is to try to debug it by setting a breakpoint there > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757783#comment-16757783 ] Uwe Schindler commented on SOLR-9515: - More info here: https://docs.oracle.com/javase/8/docs/technotes/guides/security/troubleshooting-security.html > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757782#comment-16757782
]
Uwe Schindler commented on SOLR-9515:
-
The only thing that might be an issue here: Maybe Hadoop does some separate
permission stuff which interferes with Lucene. In code you can always further
restrict permissions (we have a test using that feature) in some
AccessController-closure.
I'd suggest to enable logging of permission checks:
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/envvars003.html,
e.g. {{-Djava.security.debug="access,failure"}}
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757779#comment-16757779 ] Kevin Risden commented on SOLR-9515: Yea I have been trying to debug it. I'll keep playing with it. Thanks. Glad it isn't something extremely obvious that I'm missing. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757776#comment-16757776
]
Kevin Risden commented on SOLR-9515:
Both of the following still exhibit issues:
{code:java}
ant test -Dtestcase=HdfsRecoverLeaseTest -Dtests.seed=B3EBC148FC827CD8
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-IE
-Dtests.timezone=Asia/Ulan_Bator -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
-Dargs='-Djdk.io.permissionsUseCanonicalPath=true
-Djdk.security.filePermCompat=true'
{code}
{code:java}
ant test -Dtestcase=HdfsRecoverLeaseTest -Dtests.seed=B3EBC148FC827CD8
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-IE
-Dtests.timezone=Asia/Ulan_Bator -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1 -Djdk.io.permissionsUseCanonicalPath=true
-Djdk.security.filePermCompat=true
{code}
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757768#comment-16757768 ] Uwe Schindler commented on SOLR-9515: - In Java9 they changed the "normalization" of paths when the FilePermissions are compared. There is no longer the canonical path be calculated (for performance reasons, so the path is only made absolute but not canonicalized - if there is a symlink bad things may happen). Not sure what exactly is happening here, to me it also looks fine. Did you test this locally with Java 9+ during your development or did you only use Java 8? > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757773#comment-16757773 ] Kevin Risden commented on SOLR-9515: I had tested with just JDK8. This does reproduce locally though with JDK11. I checked the failure locally and don't see any symlinks in the path going down. I can revert the master commit since I don't have this working on JDK9+ yet. Sigh. I was going to try a few configs from here: https://docs.oracle.com/javase/10/security/permissions-jdk1.htm#JSSEC-GUID-83063225-0ACB-4909-9BAB-7F7D4E3749E2 > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757746#comment-16757746
]
Kevin Risden commented on SOLR-9515:
So this caused failures on JDK9+ not sure how the below is possible
currently since solr-tests.policy allows read access to that path.
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23605
{code:java}
[junit4] 2> java.io.IOException: Failed to start sub tasks to add replica in
replica map :java.security.AccessControlException: access denied
("java.io.FilePermission"
"/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.index.hdfs.CheckHdfsIndexTest_B3EBC148FC827CD8-001/tempDir-001/hdfsBaseDir/data/data3/current/BP-669531916-88.99.242.108-1548970895105/current/finalized"
"read")
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.getVolumeMap(BlockPoolSlice.java:439)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.getVolumeMap(FsVolumeImpl.java:1003)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeList$1.run(FsVolumeList.java:201)
~[hadoop-hdfs-3.2.0.jar:?]
{code}
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Blocker
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757567#comment-16757567 ] ASF subversion and git services commented on SOLR-9515: --- Commit 6bb24673f422a4e4267bc22361bc9258809d5f60 in lucene-solr's branch refs/heads/master from Mark Robert Miller [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6bb2467 ] SOLR-9515: Update to Hadoop 3 Signed-off-by: Kevin Risden > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4.5h > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757559#comment-16757559 ] Kevin Risden commented on SOLR-9515: Last patch is squashed commits from PR plus CHANGES.txt. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Blocker > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 4h > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756584#comment-16756584 ] Kevin Risden commented on SOLR-9515: Latest patch fixes the forbiddenapis warning. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 3.5h > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756574#comment-16756574
]
Lucene/Solr QA commented on SOLR-9515:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} Check forbidden APIs {color} | {color:red}
0m 2s{color} | {color:red} Check forbidden APIs check-forbidden-apis failed
{color} |
| {color:red}-1{color} | {color:red} Check licenses {color} | {color:red} 0m
2s{color} | {color:red} Check forbidden APIs check-forbidden-apis failed
{color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
1m 19s{color} | {color:green} Check licenses check-licenses passed {color} |
| {color:red}-1{color} | {color:red} Validate source patterns {color} |
{color:red} 0m 2s{color} | {color:red} Check forbidden APIs
check-forbidden-apis failed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
15s{color} | {color:green} tools in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 46s{color}
| {color:red} core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
33s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-9515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956948/SOLR-9515.patch |
| Optional Tests | checklicenses validatesourcepatterns ratsources compile
javac unit checkforbiddenapis |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 2beb853 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | 1.8.0_191 |
| Check forbidden APIs |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/artifact/out/patch-check-forbidden-apis-root.txt
|
| Check licenses |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/artifact/out/patch-check-forbidden-apis-root.txt
|
| Validate source patterns |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/artifact/out/patch-check-forbidden-apis-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/testReport/ |
| modules | C: lucene lucene/tools solr solr/core solr/test-framework U: . |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/281/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Major
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch,
> SOLR-9515.patch
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail:
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756548#comment-16756548 ] David Smiley commented on SOLR-9515: It's nice to see that with this we no longer have the legacy Jackson > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 3.5h > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756513#comment-16756513 ] Kevin Risden commented on SOLR-9515: Added assumption check to HdfsTestUtil to avoid not run the test if FastDateFormat fails with current locale. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch, > SOLR-9515.patch > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756413#comment-16756413 ] Jason Gerlowski commented on SOLR-9515: --- Took a quick look. I see the biggest part of the patch (other than license changes) is the HttpServer2 class you added. But I couldn't trace out how HttpServer2 gets invoked. Nothing calls the Builder in that class, AFAICT. What am I missing? Other than that question, everything looks good so far to me at least. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756430#comment-16756430 ] Kevin Risden commented on SOLR-9515: [~gerlowskija] there are some more details on the PR in comments about HttpServer2. Near the bottom here: [https://github.com/apache/lucene-solr/pull/553] Gist: Lucene in integration tests spins up a Hadoop cluster which HttpServer2 (from Hadoop internally). It only works on Jetty 9.3 but we have Jetty 9.4. So copied/patched HttpServer2 to make the integration tests work. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756401#comment-16756401
]
Kevin Risden commented on SOLR-9515:
The precommit failures from above for Hadoop are real but they are an issue
with commons-lang3. I just sent a message to the commons-lang3 mailing list. I
have seen these errors sporadically as well. The stacktrace that is similar is:
{code:java}
ava.lang.ArrayIndexOutOfBoundsException: 4
[junit4] 2>at
org.apache.commons.lang3.time.FastDatePrinter$TextField.appendTo(FastDatePrinter.java:901)
~[commons-lang3-3.7.jar:3.7]
[junit4] 2>at
org.apache.commons.lang3.time.FastDatePrinter.applyRules(FastDatePrinter.java:573)
~[commons-lang3-3.7.jar:3.7]
[junit4] 2>at
org.apache.commons.lang3.time.FastDatePrinter.applyRulesToString(FastDatePrinter.java:455)
~[commons-lang3-3.7.jar:3.7]
[junit4] 2>at
org.apache.commons.lang3.time.FastDatePrinter.format(FastDatePrinter.java:446)
~[commons-lang3-3.7.jar:3.7]
[junit4] 2>at
org.apache.commons.lang3.time.FastDateFormat.format(FastDateFormat.java:428)
~[commons-lang3-3.7.jar:3.7]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.start(DirectoryScanner.java:281)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.DataNode.initDirectoryScanner(DataNode.java:1090)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1686)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:390)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:280)
~[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:819)
[hadoop-hdfs-3.2.0.jar:?]
[junit4] 2>at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
{code}
This happens when the locale is set. I haven't nailed down which locales all
fail but a simple reproducing test case without Hadoop or Lucene/Solr is here:
{code:java}
long timestamp = System.currentTimeMillis();
Locale.setDefault(Locale.forLanguageTag("ja-JP-u-ca-japanese-x-lvariant-JP"));
Assert.assertEquals(SimpleDateFormat.getInstance().format(timestamp),
FastDateFormat.getInstance().format(timestamp));
{code}
This is with commons-lang3 3.8.1 the latest release.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Major
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756351#comment-16756351 ] Kevin Risden commented on SOLR-9515: Added missing dependencies like re2j for Hadoop on Kerberos. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[
https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756336#comment-16756336
]
Lucene/Solr QA commented on SOLR-9515:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check licenses {color} | {color:green}
0m 23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
8s{color} | {color:green} tools in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 35s{color}
| {color:red} core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.index.hdfs.CheckHdfsIndexTest |
| | solr.update.TestHdfsUpdateLog |
| | solr.cloud.LeaderTragicEventTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-9515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956719/SOLR-9515.patch |
| Optional Tests | checklicenses validatesourcepatterns ratsources compile
javac unit checkforbiddenapis |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 57f693b |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/278/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/278/testReport/ |
| modules | C: lucene lucene/tools solr solr/core solr/test-framework U: . |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/278/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Update to Hadoop 3
> --
>
> Key: SOLR-9515
> URL: https://issues.apache.org/jira/browse/SOLR-9515
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Kevin Risden
>Priority: Major
> Fix For: 8.0, master (9.0)
>
> Attachments: SOLR-9515.patch, SOLR-9515.patch
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared.
> I'll start up a dev branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755060#comment-16755060 ] Kevin Risden commented on SOLR-9515: Updated patch to master with HDFS tests passing. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Kevin Risden >Priority: Major > Fix For: 8.0, master (9.0) > > Attachments: SOLR-9515.patch, SOLR-9515.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752374#comment-16752374 ] Kevin Risden commented on SOLR-9515: I'm hoping to take a crack at this and get it into Solr 8.x. I know I am a bit late to the 8.x party but Hadoop 3.x will be needed for full JDK 11 support with Solr on HDFS. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > Attachments: SOLR-9515.patch > > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973640#comment-15973640 ] Hrishikesh Gadre commented on SOLR-9515: [~markrmil...@gmail.com] Should we create sub-tasks as we identify new issues? e.g. after alpha1, alpha2 etc. That will keep individual patches simpler to review and apply. I have already done this for security related changes (Ref: SOLR-9761). > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller > Attachments: SOLR-9515.patch > > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940790#comment-15940790 ] Steve Rowe commented on SOLR-9515: -- The kite/morphlines/map-reduce contribs were removed in SOLR-9221 , which will affect the patch here greatly. > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller > Attachments: SOLR-9515.patch > > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9515) Update to Hadoop 3
[ https://issues.apache.org/jira/browse/SOLR-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15500386#comment-15500386 ] Mark Miller commented on SOLR-9515: --- Pushed a branch to jira/solr-9515 > Update to Hadoop 3 > -- > > Key: SOLR-9515 > URL: https://issues.apache.org/jira/browse/SOLR-9515 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller > Attachments: SOLR-9515.patch > > > Hadoop 3 is not out yet, but I'd like to iron out the upgrade to be prepared. > I'll start up a dev branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
