[jira] [Created] (HADOOP-15112) create-release didn't sign artifacts
Andrew Wang created HADOOP-15112: Summary: create-release didn't sign artifacts Key: HADOOP-15112 URL: https://issues.apache.org/jira/browse/HADOOP-15112 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Reporter: Andrew Wang While building the 3.0.0 RC1, I had to re-invoke Maven because the create-release script didn't deploy signatures to Nexus. Looking at the repo (and my artifacts), it seems like "sign" didn't run properly. I lost my create-release output, but I noticed that it will log and continue rather than abort in some error conditions. This might have caused my lack of signatures. IMO it'd be better to explicitly fail in these situations. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15058) create-release site build outputs dummy shaded jars due to skipShade
Andrew Wang created HADOOP-15058: Summary: create-release site build outputs dummy shaded jars due to skipShade Key: HADOOP-15058 URL: https://issues.apache.org/jira/browse/HADOOP-15058 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Blocker -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15037) Add site release notes for OrgQueue and resource types
Andrew Wang created HADOOP-15037: Summary: Add site release notes for OrgQueue and resource types Key: HADOOP-15037 URL: https://issues.apache.org/jira/browse/HADOOP-15037 Project: Hadoop Common Issue Type: Improvement Reporter: Andrew Wang Assignee: Andrew Wang Let's add some small blurbs and doc links to the site release notes for these features. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15018) Update JAVA_HOME in create-release for Xenial Dockerfile
Andrew Wang created HADOOP-15018: Summary: Update JAVA_HOME in create-release for Xenial Dockerfile Key: HADOOP-15018 URL: https://issues.apache.org/jira/browse/HADOOP-15018 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 3.0.0 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Blocker create-release expects the Oracle JDK when setting JAVA_HOME. HADOOP-14816 no longer includes the Oracle JDK, so we need to update this to point to OpenJDK instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14555) document how to run wasb tests in azure docs site/testing.md
[ https://issues.apache.org/jira/browse/HADOOP-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14555. -- Resolution: Duplicate Fix Version/s: (was: 3.0.0) > document how to run wasb tests in azure docs site/testing.md > > > Key: HADOOP-14555 > URL: https://issues.apache.org/jira/browse/HADOOP-14555 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 2.8.1 >Reporter: Steve Loughran > > There's no single (current) documentation on running the azure tests > * There's some in site/index.md, but iit looks potentially out of date > (refers to an older azure SDK version) > * There's a file > {{src/test/org/apache/hadoop/fs/azure/RunningLiveWasbTests.txt}} which > refers to a nonexistent doc {{hadoop-tools/hadoop-azure/README.txt }} for > instructions. > Proposed: > # move testing docs out of main azure doc page, with link from there. > # bring up to date with SDK, move of tests to ITests. > # purge all other references, including bits of test javadocs which are no > longer correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14555) document how to run wasb tests in azure docs site/testing.md
[ https://issues.apache.org/jira/browse/HADOOP-14555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14555: -- > document how to run wasb tests in azure docs site/testing.md > > > Key: HADOOP-14555 > URL: https://issues.apache.org/jira/browse/HADOOP-14555 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 2.8.1 >Reporter: Steve Loughran > > There's no single (current) documentation on running the azure tests > * There's some in site/index.md, but iit looks potentially out of date > (refers to an older azure SDK version) > * There's a file > {{src/test/org/apache/hadoop/fs/azure/RunningLiveWasbTests.txt}} which > refers to a nonexistent doc {{hadoop-tools/hadoop-azure/README.txt }} for > instructions. > Proposed: > # move testing docs out of main azure doc page, with link from there. > # bring up to date with SDK, move of tests to ITests. > # purge all other references, including bits of test javadocs which are no > longer correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14928) Update site release notes for 3.0.0 GA
Andrew Wang created HADOOP-14928: Summary: Update site release notes for 3.0.0 GA Key: HADOOP-14928 URL: https://issues.apache.org/jira/browse/HADOOP-14928 Project: Hadoop Common Issue Type: Improvement Components: site Affects Versions: 3.0.0 Reporter: Andrew Wang Assignee: Andrew Wang We should update the site release notes blurb for 3.0.0 GA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13917) Ensure yetus personality runs the integration tests for the shaded client
[ https://issues.apache.org/jira/browse/HADOOP-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13917: -- > Ensure yetus personality runs the integration tests for the shaded client > - > > Key: HADOOP-13917 > URL: https://issues.apache.org/jira/browse/HADOOP-13917 > Project: Hadoop Common > Issue Type: Sub-task > Components: build, test >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-13917.WIP.0.patch, HADOOP-14771.02.patch > > > Either QBT or a different jenkins job should run our integration tests, > specifically the ones added for the shaded client. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13917) Ensure yetus personality runs the integration tests for the shaded client
[ https://issues.apache.org/jira/browse/HADOOP-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13917. -- Resolution: Delivered > Ensure yetus personality runs the integration tests for the shaded client > - > > Key: HADOOP-13917 > URL: https://issues.apache.org/jira/browse/HADOOP-13917 > Project: Hadoop Common > Issue Type: Sub-task > Components: build, test >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-13917.WIP.0.patch, HADOOP-14771.02.patch > > > Either QBT or a different jenkins job should run our integration tests, > specifically the ones added for the shaded client. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14545) Uninitialized S3A instance NPEs on toString()
[ https://issues.apache.org/jira/browse/HADOOP-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14545. -- Resolution: Duplicate > Uninitialized S3A instance NPEs on toString() > - > > Key: HADOOP-14545 > URL: https://issues.apache.org/jira/browse/HADOOP-14545 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.0.0-beta1 > > > You can't log an uninited S3AFileSystem instance without getting a stack trace > {code} > java.lang.NullPointerException > at > org.apache.hadoop.fs.s3a.S3AFileSystem.getDefaultBlockSize(S3AFileSystem.java:2131) > at > org.apache.hadoop.fs.s3a.S3AFileSystem.toString(S3AFileSystem.java:2148) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14545) Uninitialized S3A instance NPEs on toString()
[ https://issues.apache.org/jira/browse/HADOOP-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14545: -- > Uninitialized S3A instance NPEs on toString() > - > > Key: HADOOP-14545 > URL: https://issues.apache.org/jira/browse/HADOOP-14545 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.0.0-beta1 > > > You can't log an uninited S3AFileSystem instance without getting a stack trace > {code} > java.lang.NullPointerException > at > org.apache.hadoop.fs.s3a.S3AFileSystem.getDefaultBlockSize(S3AFileSystem.java:2131) > at > org.apache.hadoop.fs.s3a.S3AFileSystem.toString(S3AFileSystem.java:2148) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14834) Remove original S3A output stream
[ https://issues.apache.org/jira/browse/HADOOP-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14834. -- Resolution: Duplicate > Remove original S3A output stream > - > > Key: HADOOP-14834 > URL: https://issues.apache.org/jira/browse/HADOOP-14834 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.0.0-beta1 > > > The S3A Block output stream is working well and much better than the original > stream in terms of: scale, performance, instrumentation, robustness > Proposed: switch this to be the default, as a precursor to removing it later > HADOOP-14746 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14834) Remove original S3A output stream
[ https://issues.apache.org/jira/browse/HADOOP-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14834: -- > Remove original S3A output stream > - > > Key: HADOOP-14834 > URL: https://issues.apache.org/jira/browse/HADOOP-14834 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.0.0-beta1 > > > The S3A Block output stream is working well and much better than the original > stream in terms of: scale, performance, instrumentation, robustness > Proposed: switch this to be the default, as a precursor to removing it later > HADOOP-14746 -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14879) Build failure due to failing hadoop-client-check-invariants for hadoop-client-runtime.jar
[ https://issues.apache.org/jira/browse/HADOOP-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14879. -- Resolution: Done > Build failure due to failing hadoop-client-check-invariants for > hadoop-client-runtime.jar > - > > Key: HADOOP-14879 > URL: https://issues.apache.org/jira/browse/HADOOP-14879 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.1.0 >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Blocker > Fix For: 3.0.0-beta1 > > > {noformat} > [ERROR] Found artifact with unexpected contents: > '/.../hadoop-client-modules/hadoop-client-runtime/target/hadoop-client-runtime-3.1.0-SNAPSHOT.jar' > Please check the following and either correct the build or update > the allowed list with reasoning. > javax/ > javax/inject/ > javax/inject/Inject.class > javax/inject/Named.class > javax/inject/Provider.class > javax/inject/Qualifier.class > javax/inject/Scope.class > javax/inject/Singleton.class > jersey/ > jersey/repackaged/ > jersey/repackaged/org/ > jersey/repackaged/org/objectweb/ > jersey/repackaged/org/objectweb/asm/ > jersey/repackaged/org/objectweb/asm/AnnotationVisitor.class > jersey/repackaged/org/objectweb/asm/AnnotationWriter.class > jersey/repackaged/org/objectweb/asm/Attribute.class > jersey/repackaged/org/objectweb/asm/ByteVector.class > jersey/repackaged/org/objectweb/asm/ClassReader.class > jersey/repackaged/org/objectweb/asm/ClassVisitor.class > jersey/repackaged/org/objectweb/asm/ClassWriter.class > jersey/repackaged/org/objectweb/asm/Context.class > jersey/repackaged/org/objectweb/asm/Edge.class > jersey/repackaged/org/objectweb/asm/FieldVisitor.class > jersey/repackaged/org/objectweb/asm/FieldWriter.class > jersey/repackaged/org/objectweb/asm/Frame.class > jersey/repackaged/org/objectweb/asm/Handle.class > jersey/repackaged/org/objectweb/asm/Handler.class > jersey/repackaged/org/objectweb/asm/Item.class > jersey/repackaged/org/objectweb/asm/Label.class > jersey/repackaged/org/objectweb/asm/MethodVisitor.class > jersey/repackaged/org/objectweb/asm/MethodWriter.class > jersey/repackaged/org/objectweb/asm/Opcodes.class > jersey/repackaged/org/objectweb/asm/Type.class > jersey/repackaged/org/objectweb/asm/TypePath.class > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14879) Build failure due to failing hadoop-client-check-invariants for hadoop-client-runtime.jar
[ https://issues.apache.org/jira/browse/HADOOP-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14879: -- > Build failure due to failing hadoop-client-check-invariants for > hadoop-client-runtime.jar > - > > Key: HADOOP-14879 > URL: https://issues.apache.org/jira/browse/HADOOP-14879 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.1.0 >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Blocker > Fix For: 3.0.0-beta1 > > > {noformat} > [ERROR] Found artifact with unexpected contents: > '/.../hadoop-client-modules/hadoop-client-runtime/target/hadoop-client-runtime-3.1.0-SNAPSHOT.jar' > Please check the following and either correct the build or update > the allowed list with reasoning. > javax/ > javax/inject/ > javax/inject/Inject.class > javax/inject/Named.class > javax/inject/Provider.class > javax/inject/Qualifier.class > javax/inject/Scope.class > javax/inject/Singleton.class > jersey/ > jersey/repackaged/ > jersey/repackaged/org/ > jersey/repackaged/org/objectweb/ > jersey/repackaged/org/objectweb/asm/ > jersey/repackaged/org/objectweb/asm/AnnotationVisitor.class > jersey/repackaged/org/objectweb/asm/AnnotationWriter.class > jersey/repackaged/org/objectweb/asm/Attribute.class > jersey/repackaged/org/objectweb/asm/ByteVector.class > jersey/repackaged/org/objectweb/asm/ClassReader.class > jersey/repackaged/org/objectweb/asm/ClassVisitor.class > jersey/repackaged/org/objectweb/asm/ClassWriter.class > jersey/repackaged/org/objectweb/asm/Context.class > jersey/repackaged/org/objectweb/asm/Edge.class > jersey/repackaged/org/objectweb/asm/FieldVisitor.class > jersey/repackaged/org/objectweb/asm/FieldWriter.class > jersey/repackaged/org/objectweb/asm/Frame.class > jersey/repackaged/org/objectweb/asm/Handle.class > jersey/repackaged/org/objectweb/asm/Handler.class > jersey/repackaged/org/objectweb/asm/Item.class > jersey/repackaged/org/objectweb/asm/Label.class > jersey/repackaged/org/objectweb/asm/MethodVisitor.class > jersey/repackaged/org/objectweb/asm/MethodWriter.class > jersey/repackaged/org/objectweb/asm/Opcodes.class > jersey/repackaged/org/objectweb/asm/Type.class > jersey/repackaged/org/objectweb/asm/TypePath.class > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11656) Classpath isolation for downstream clients
[ https://issues.apache.org/jira/browse/HADOOP-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11656. -- Resolution: Done Hadoop Flags: (was: Incompatible change) Fix Version/s: 3.0.0-beta1 I'm resolving this as Done since all subtasks have been completed. There are still a few follow-ons being tracked for 3.0.0 GA. Many thanks to Sean for driving this forward! Great work! > Classpath isolation for downstream clients > -- > > Key: HADOOP-11656 > URL: https://issues.apache.org/jira/browse/HADOOP-11656 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Labels: classloading, classpath, dependencies, scripts, shell > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-11656_proposal.md > > > Currently, Hadoop exposes downstream clients to a variety of third party > libraries. As our code base grows and matures we increase the set of > libraries we rely on. At the same time, as our user base grows we increase > the likelihood that some downstream project will run into a conflict while > attempting to use a different version of some library we depend on. This has > already happened with i.e. Guava several times for HBase, Accumulo, and Spark > (and I'm sure others). > While YARN-286 and MAPREDUCE-1700 provided an initial effort, they default to > off and they don't do anything to help dependency conflicts on the driver > side or for folks talking to HDFS directly. This should serve as an umbrella > for changes needed to do things thoroughly on the next major version. > We should ensure that downstream clients > 1) can depend on a client artifact for each of HDFS, YARN, and MapReduce that > doesn't pull in any third party dependencies > 2) only see our public API classes (or as close to this as feasible) when > executing user provided code, whether client side in a launcher/driver or on > the cluster in a container or within MR. > This provides us with a double benefit: users get less grief when they want > to run substantially ahead or behind the versions we need and the project is > freer to change our own dependency versions because they'll no longer be in > our compatibility promises. > Project specific task jiras to follow after I get some justifying use cases > written in the comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14655) Update httpcore version to 4.4.6
[ https://issues.apache.org/jira/browse/HADOOP-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14655: -- Reverted this JIRA from trunk and branch-3.0 per Marton's instructions. > Update httpcore version to 4.4.6 > > > Key: HADOOP-14655 > URL: https://issues.apache.org/jira/browse/HADOOP-14655 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Ray Chiang >Assignee: Ray Chiang > Attachments: HADOOP-14655.001.patch > > > Update the dependency > org.apache.httpcomponents:httpcore:4.4.4 > to the latest (4.4.6). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14799) Update nimbus-jose-jwt to 4.41.1
[ https://issues.apache.org/jira/browse/HADOOP-14799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14799. -- Resolution: Fixed Let's re-resolve and track the follow-on work in another JIRA. Thanks Ray, Steve. > Update nimbus-jose-jwt to 4.41.1 > > > Key: HADOOP-14799 > URL: https://issues.apache.org/jira/browse/HADOOP-14799 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Ray Chiang >Assignee: Ray Chiang > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-14799.001.patch, HADOOP-14799.002.patch, > HADOOP-14799.003.patch > > > Update the dependency > com.nimbusds:nimbus-jose-jwt:3.9 > to the latest (4.41.1) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14655) Update httpcore version to 4.4.6
[ https://issues.apache.org/jira/browse/HADOOP-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14655. -- Resolution: Fixed While it's in the branch, let's leave the JIRA resolved for release notes purposes. > Update httpcore version to 4.4.6 > > > Key: HADOOP-14655 > URL: https://issues.apache.org/jira/browse/HADOOP-14655 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Ray Chiang >Assignee: Ray Chiang > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-14655.001.patch > > > Update the dependency > org.apache.httpcomponents:httpcore:4.4.4 > to the latest (4.4.6). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14848) Switch from JDiff to japicmp
Andrew Wang created HADOOP-14848: Summary: Switch from JDiff to japicmp Key: HADOOP-14848 URL: https://issues.apache.org/jira/browse/HADOOP-14848 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha4 Reporter: Andrew Wang JDiff is old and not maintained. It complicates our build by requiring xerces, and also a lot of Maven logic to custom patch it and stitch it up. japicmp was proposed as a more up-to-date tool that also has a Maven plugin. It's also ALv2 which is a nice bonus. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13998. -- Resolution: Done Re-resolving per above. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13998: -- Re-opening to resolve as "Complete" or something, since this code change was attributed to the parent JIRA HADOOP-13345 in the commit message. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14674) Correct javadoc for getRandomizedTempPath
[ https://issues.apache.org/jira/browse/HADOOP-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-14674: -- I don't see this showing up in trunk, did it actually get committed? > Correct javadoc for getRandomizedTempPath > - > > Key: HADOOP-14674 > URL: https://issues.apache.org/jira/browse/HADOOP-14674 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Mukul Kumar Singh >Assignee: Mukul Kumar Singh > Attachments: HADOOP-14674.001.patch > > > getRandomizedTempPath has incorrect javadoc where the javadoc specifies a > parameter to the function however the function doesnt expects one. > {code} > /** >* Get a temp path. This may or may not be relative; it depends on what the >* {@link #SYSPROP_TEST_DATA_DIR} is set to. If unset, it returns a path >* under the relative path {@link #DEFAULT_TEST_DATA_PATH} >* @param subpath sub path, with no leading "/" character >* @return a string to use in paths >*/ > public static String getRandomizedTempPath() { > return getTempPath(RandomStringUtils.randomAlphanumeric(10)); > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14805) Upgrade to zstd 1.3.1
Andrew Wang created HADOOP-14805: Summary: Upgrade to zstd 1.3.1 Key: HADOOP-14805 URL: https://issues.apache.org/jira/browse/HADOOP-14805 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha2, 2.9.0 Reporter: Andrew Wang -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14731) Update gitignore to exclude output of site build
Andrew Wang created HADOOP-14731: Summary: Update gitignore to exclude output of site build Key: HADOOP-14731 URL: https://issues.apache.org/jira/browse/HADOOP-14731 Project: Hadoop Common Issue Type: Improvement Components: build, site Affects Versions: 3.0.0-alpha3 Reporter: Andrew Wang Assignee: Andrew Wang Site build generates a bunch of files that aren't caught by gitignore, let's update. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14501) aalto-xml cannot handle some odd XML features
Andrew Wang created HADOOP-14501: Summary: aalto-xml cannot handle some odd XML features Key: HADOOP-14501 URL: https://issues.apache.org/jira/browse/HADOOP-14501 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 2.9.0, 3.0.0-alpha4 Reporter: Andrew Wang Priority: Blocker [~hgadre] tried testing solr with a Hadoop 3 client. He saw various test case failures due to what look like functionality gaps in the new aalto-xml stax implementation pulled in by HADOOP-14216: {noformat} [junit4]> Throwable #1: com.fasterxml.aalto.WFCException: Illegal XML character ('ΓΌ' (code 252)) [junit4]> Caused by: com.fasterxml.aalto.WFCException: General entity reference (&bar;) encountered in entity expanding mode: operation not (yet) implemented ... [junit4]> Throwable #1: org.apache.solr.common.SolrException: General entity reference (&wacky;) encountered in entity expanding mode: operation not (yet) implemented {noformat} These were from the following test case executions: {noformat} NOTE: reproduce with: ant test -Dtestcase=DocumentAnalysisRequestHandlerTest -Dtests.method=testCharsetOutsideDocument -Dtests.seed=2F739D88D9C723CA -Dtests.slow=true -Dtests.locale=und -Dtests.timezone=Atlantic/Faeroe -Dtests.asserts=true -Dtests.file.encoding=US-ASCII NOTE: reproduce with: ant test -Dtestcase=MBeansHandlerTest -Dtests.method=testXMLDiffWithExternalEntity -Dtests.seed=2F739D88D9C723CA -Dtests.slow=true -Dtests.locale=en-US -Dtests.timezone=US/Aleutian -Dtests.asserts=true -Dtests.file.encoding=US-ASCII NOTE: reproduce with: ant test -Dtestcase=XmlUpdateRequestHandlerTest -Dtests.method=testExternalEntities -Dtests.seed=2F739D88D9C723CA -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=America/Barbados -Dtests.asserts=true -Dtests.file.encoding=US-ASCII NOTE: reproduce with: ant test -Dtestcase=XmlUpdateRequestHandlerTest -Dtests.method=testNamedEntity -Dtests.seed=2F739D88D9C723CA -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=America/Barbados -Dtests.asserts=true -Dtests.file.encoding=US-ASCII {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14399) Configuration does not correctly XInclude absolute file URIs
Andrew Wang created HADOOP-14399: Summary: Configuration does not correctly XInclude absolute file URIs Key: HADOOP-14399 URL: https://issues.apache.org/jira/browse/HADOOP-14399 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 2.9.0, 3.0.0-alpha3 Reporter: Andrew Wang Priority: Blocker [Reported by|https://issues.apache.org/jira/browse/HADOOP-14216?focusedCommentId=15967816&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15967816] [~ste...@apache.org] on HADOOP-14216, filing this JIRA on his behalf: {quote} Just tracked this down as the likely cause of my S3A test failures. This is pulling in core-site.xml, which then xincludes auth-keys.xml, which finally references an absolute path, file://home/stevel/(secret)/aws-keys.xml. This is failing for me even with the latest patch in. Either transient XIncludes aren't being picked up or Note also I think the error could be improved. 1. It's in the included file where the problem appears to lie and 2. we should really know the missing entry. Perhaps a wiki link too: I had to read the XInclude spec to work out what was going on here before I could go back to finding the cause {quote} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14216) Improve Configuration XML Parsing Performance
[ https://issues.apache.org/jira/browse/HADOOP-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-14216. -- Resolution: Fixed I'm going to re-resolving this so the release notes are consistent. If we revert this JIRA, then it's appropriate to re-open. I'll file a follow-on issue for tracking. > Improve Configuration XML Parsing Performance > - > > Key: HADOOP-14216 > URL: https://issues.apache.org/jira/browse/HADOOP-14216 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14216.1.patch, HADOOP-14216.2-branch-2.patch, > HADOOP-14216.2.patch, HADOOP-14216.addendum.1.patch > > > JIRA is to improve XML parsing performance through reuse and a change in XML > parser (STAX) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14386) Make trunk work with Guava 11.0.2 again
Andrew Wang created HADOOP-14386: Summary: Make trunk work with Guava 11.0.2 again Key: HADOOP-14386 URL: https://issues.apache.org/jira/browse/HADOOP-14386 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha3 Reporter: Andrew Wang As an alternative to reverting or shading HADOOP-10101 (the upgrade of Guava from 11.0.2 to 21.0), HADOOP-14380 makes the Guava version configurable. However, it still doesn't compile with Guava 11.0.2, since HADOOP-10101 chose to use the moved Guava classes rather than replacing them with alternatives. This JIRA aims to make Hadoop compatible with Guava 11.0.2 as well as 21.0 by replacing usage of these moved Guava classes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14382) Remove usages of MoreObjects.toStringHelper
Andrew Wang created HADOOP-14382: Summary: Remove usages of MoreObjects.toStringHelper Key: HADOOP-14382 URL: https://issues.apache.org/jira/browse/HADOOP-14382 Project: Hadoop Common Issue Type: Improvement Components: metrics Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Minor MoreObjects.toStringHelper is a source of incompatibility across Guava versions. Let's move off of this to a native Java 8 API. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-10604) CryptoFileSystem decorator using xAttrs and KeyProvider
[ https://issues.apache.org/jira/browse/HADOOP-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-10604. -- Resolution: Incomplete I think we decided not to pursue this effort further, and focused instead on HDFS transparent encryption. Resolving. > CryptoFileSystem decorator using xAttrs and KeyProvider > --- > > Key: HADOOP-10604 > URL: https://issues.apache.org/jira/browse/HADOOP-10604 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134) >Reporter: Alejandro Abdelnur >Assignee: Yi Liu > Attachments: HADOOP-10604.1.patch, HADOOP-10604.patch > > > A FileSystem implementation that wraps an existing filesystem and provides > encryption. It will require the underlying filesystem to support xAttrs. It > will use the KeyProvider API to retrieve encryption keys. > This is mostly the work in the patch HADOOP-10150 minus the crypto streams -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14293) Initialize FakeTimer with a less trivial value.
Andrew Wang created HADOOP-14293: Summary: Initialize FakeTimer with a less trivial value. Key: HADOOP-14293 URL: https://issues.apache.org/jira/browse/HADOOP-14293 Project: Hadoop Common Issue Type: Bug Components: test Affects Versions: 3.0.0-alpha3 Reporter: Andrew Wang Assignee: Andrew Wang HADOOP-14276 broke TestFsDatasetImpl#testLoadingDfsUsedForVolumes which uses a FakeTimer. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14284) Shade Guava everywhere
Andrew Wang created HADOOP-14284: Summary: Shade Guava everywhere Key: HADOOP-14284 URL: https://issues.apache.org/jira/browse/HADOOP-14284 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0-alpha3 Reporter: Andrew Wang Priority: Blocker HADOOP-10101 upgraded the guava version for 3.x to 21. Guava is broadly used by Java projects that consume our artifacts. Unfortunately, these projects also consume our private artifacts like {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced by HADOOP-11804, currently only available in 3.0.0-alpha2. We should shade Guava everywhere to proactively avoid breaking downstreams. This isn't a requirement for all dependency upgrades, but it's necessary for known-bad dependencies like Guava. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14280) Fix compilation of TestKafkaMetrics
Andrew Wang created HADOOP-14280: Summary: Fix compilation of TestKafkaMetrics Key: HADOOP-14280 URL: https://issues.apache.org/jira/browse/HADOOP-14280 Project: Hadoop Common Issue Type: Improvement Components: tools Affects Versions: 3.0.0-alpha3 Reporter: Andrew Wang Assignee: Andrew Wang The upgrade to Guava 21 at HADOOP-10101 missed updating this Kafka test. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14257) hadoop-auth and hadoop-annotations jars are in lib directory
Andrew Wang created HADOOP-14257: Summary: hadoop-auth and hadoop-annotations jars are in lib directory Key: HADOOP-14257 URL: https://issues.apache.org/jira/browse/HADOOP-14257 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 3.0.0-alpha2, 2.8.0 Reporter: Andrew Wang Poking around in the 3.0.0-alpha2 tarball, noticed that the auth and annotations JARs seem to be in the wrong place (lib dir): {noformat} ./share/hadoop/common/lib/hadoop-annotations-3.0.0-alpha2.jar ./share/hadoop/common/lib/hadoop-auth-3.0.0-alpha2.jar {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-9357) Fallback to default authority if not specified in FileContext
[ https://issues.apache.org/jira/browse/HADOOP-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-9357. - Resolution: Invalid I'm going to just resolve this one, unlikely we're going to make any FileContext changes at this point. > Fallback to default authority if not specified in FileContext > - > > Key: HADOOP-9357 > URL: https://issues.apache.org/jira/browse/HADOOP-9357 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.0.0-alpha >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Attachments: hadoop-9357-1.patch, hadoop-9357-2.patch, > hadoop-9357-3.patch, hadoop-9357-testfixup.patch > > > Currently, FileContext adheres rather strictly to RFC2396 when it comes to > parsing absolute URIs (URIs with a scheme). If a user asks for a URI like > "hdfs:///tmp", FileContext will error while FileSystem will add the authority > of the default FS (e.g. turn it into "hdfs://defaultNN:port/tmp"). > This is technically correct, but FileSystem's behavior is nicer for users and > okay based on 5.2.3 in the RFC, so lets do it in FileContext too: > {noformat} > For backwards > compatibility, an implementation may work around such references > by removing the scheme if it matches that of the base URI and the > scheme is known to always use the syntax. The parser > can then continue with the steps below for the remainder of the > reference components. Validating parsers should mark such a > misformed relative reference as an error. > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-10856) HarFileSystem and HarFs support for HDFS encryption
[ https://issues.apache.org/jira/browse/HADOOP-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-10856. -- Resolution: Won't Fix I agree with Tucu's assessment above, we don't need to do finer-grained encryption inside a HAR. It's a file, we can treat it like a file. Resolving this old JIRA. > HarFileSystem and HarFs support for HDFS encryption > --- > > Key: HADOOP-10856 > URL: https://issues.apache.org/jira/browse/HADOOP-10856 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134) >Reporter: Andrew Wang >Assignee: Andrew Wang > > We need to examine support for Har with HDFS encryption. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14045) Aliyun OSS documentation missing from website
Andrew Wang created HADOOP-14045: Summary: Aliyun OSS documentation missing from website Key: HADOOP-14045 URL: https://issues.apache.org/jira/browse/HADOOP-14045 Project: Hadoop Common Issue Type: Bug Components: documentation, fs/oss Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang I'm looking at the alpha2 website, and can't find a link to the Aliyun OSS documentation. Under the "Hadoop Compatible File Systems" header there are links to S3, Azure blob, ADLS, and Swift, but not Aliyun OSS. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14024) KMS JMX endpoint throws ClassNotFoundException
Andrew Wang created HADOOP-14024: Summary: KMS JMX endpoint throws ClassNotFoundException Key: HADOOP-14024 URL: https://issues.apache.org/jira/browse/HADOOP-14024 Project: Hadoop Common Issue Type: Bug Components: kms Affects Versions: 2.8.0 Reporter: Andrew Wang Throws like this: {noformat} root cause java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1698) org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1544) org.apache.hadoop.jmx.JMXJsonServlet.doGet(JMXJsonServlet.java:174) javax.servlet.http.HttpServlet.service(HttpServlet.java:617) javax.servlet.http.HttpServlet.service(HttpServlet.java:723) org.apache.hadoop.crypto.key.kms.server.KMSMDCFilter.doFilter(KMSMDCFilter.java:84) org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:636) org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:304) org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:588) org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129) {noformat} I tried out branch-2.6 and it seems to be okay, so something changed between 2.6.x and 2.8.x/branch-2 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14014) Shading runs on mvn deploy
Andrew Wang created HADOOP-14014: Summary: Shading runs on mvn deploy Key: HADOOP-14014 URL: https://issues.apache.org/jira/browse/HADOOP-14014 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang I'm running "mvn deploy -DskipTests" and see that there is shading happening in the build output. This seems like a bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14003) Make additional KMS tomcat settings configurable
Andrew Wang created HADOOP-14003: Summary: Make additional KMS tomcat settings configurable Key: HADOOP-14003 URL: https://issues.apache.org/jira/browse/HADOOP-14003 Project: Hadoop Common Issue Type: Improvement Components: kms Affects Versions: 2.8.0 Reporter: Andrew Wang Assignee: Andrew Wang Doing some Tomcat performance tuning on a loaded cluster, we found that {{acceptCount}}, {{acceptorThreadCount}}, and {{protocol}} can be useful. Let's make these configurable in the kms startup script. Since the KMS is Jetty in 3.x, this is targeted at just branch-2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13996) Fix some release build issues
Andrew Wang created HADOOP-13996: Summary: Fix some release build issues Key: HADOOP-13996 URL: https://issues.apache.org/jira/browse/HADOOP-13996 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Blocker Found some build issues while doing some test runs with the create-release.sh script. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13978) Update project release notes for 3.0.0-alpha2
Andrew Wang created HADOOP-13978: Summary: Update project release notes for 3.0.0-alpha2 Key: HADOOP-13978 URL: https://issues.apache.org/jira/browse/HADOOP-13978 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang Assignee: Andrew Wang Let's update the website release notes for 3.0.0-alpha2's changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13136) shade protobuf in the hadoop-common jar
[ https://issues.apache.org/jira/browse/HADOOP-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13136. -- Resolution: Duplicate Fix Version/s: 3.0.0-alpha2 Resolving since HADOOP-11804 has been committed. > shade protobuf in the hadoop-common jar > --- > > Key: HADOOP-13136 > URL: https://issues.apache.org/jira/browse/HADOOP-13136 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Haohui Mai > Fix For: 3.0.0-alpha2 > > > While Protobuf has good wire compatibility, its implementation has been > changed from time to time. It might be a good idea to shade it in for better > compatibility. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13586) Hadoop 3.0 build broken on windows
[ https://issues.apache.org/jira/browse/HADOOP-13586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13586. -- Resolution: Cannot Reproduce I'm going to close this since Chris tested this in late October, and there haven't been any responses to requests for additional info. If we get additional info, we can reopen to track a fix. > Hadoop 3.0 build broken on windows > -- > > Key: HADOOP-13586 > URL: https://issues.apache.org/jira/browse/HADOOP-13586 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.0.0-alpha1 > Environment: Windows Server >Reporter: Steve Loughran >Priority: Blocker > > Builds on windows fail, even before getting to the native bits > Looks like dev-support/bin/dist-copynativelibs isn't windows-ready -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13663) Index out of range in SysInfoWindows
[ https://issues.apache.org/jira/browse/HADOOP-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13663. -- Resolution: Fixed Re-resolving so the status is "Fixed" rather than "Resolved". > Index out of range in SysInfoWindows > > > Key: HADOOP-13663 > URL: https://issues.apache.org/jira/browse/HADOOP-13663 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 2.7.3 > Environment: Windows >Reporter: Inigo Goiri >Assignee: Inigo Goiri > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-13663.000.patch, HADOOP-13663.001.patch > > > Sometimes, the {{NodeResourceMonitor}} tries to read the system utilization > from winutils.exe and this return empty values. This triggers the following > exception: > java.lang.StringIndexOutOfBoundsException: String index out of range: -1 > at java.lang.String.substring(String.java:1911) > at > org.apache.hadoop.util.SysInfoWindows.refreshIfNeeded(SysInfoWindows.java:158) > at > org.apache.hadoop.util.SysInfoWindows.getPhysicalMemorySize(SysInfoWindows.java:247) > at > org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getPhysicalMemorySize(ResourceCalculatorPlugin.java:63) > at > org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl$MonitoringThread.run(NodeResourceMonitorImpl.java:139) > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13663) Index out of range in SysInfoWindows
[ https://issues.apache.org/jira/browse/HADOOP-13663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13663: -- > Index out of range in SysInfoWindows > > > Key: HADOOP-13663 > URL: https://issues.apache.org/jira/browse/HADOOP-13663 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 2.7.3 > Environment: Windows >Reporter: Inigo Goiri >Assignee: Inigo Goiri > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-13663.000.patch, HADOOP-13663.001.patch > > > Sometimes, the {{NodeResourceMonitor}} tries to read the system utilization > from winutils.exe and this return empty values. This triggers the following > exception: > java.lang.StringIndexOutOfBoundsException: String index out of range: -1 > at java.lang.String.substring(String.java:1911) > at > org.apache.hadoop.util.SysInfoWindows.refreshIfNeeded(SysInfoWindows.java:158) > at > org.apache.hadoop.util.SysInfoWindows.getPhysicalMemorySize(SysInfoWindows.java:247) > at > org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getPhysicalMemorySize(ResourceCalculatorPlugin.java:63) > at > org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl$MonitoringThread.run(NodeResourceMonitorImpl.java:139) > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13423) Run JDiff on trunk for Hadoop-Common and analyze results
[ https://issues.apache.org/jira/browse/HADOOP-13423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13423. -- Resolution: Duplicate I'd like to close this in favor of the JACC reports. I checked the JACC report for alpha1, and found it easier to use than jdiff. > Run JDiff on trunk for Hadoop-Common and analyze results > > > Key: HADOOP-13423 > URL: https://issues.apache.org/jira/browse/HADOOP-13423 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Blocker > Attachments: 3.0.0-alpha1-hadoop-common-jdiff.zip, > 3.0.0-alpha1-jdiff.zip > > > We need to run JDiff and make sure the first 3.0.0 alpha release doesn't > include unnecessary API incompatible change. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13543) [Umbrella] Analyse 2.8.0 and 3.0.0-alpha1 jdiff reports and fix any issues
[ https://issues.apache.org/jira/browse/HADOOP-13543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13543. -- Resolution: Duplicate Been a month since my last comment. Seems like we can close this as dupe to the ACC work. Please reopen if you disagree. > [Umbrella] Analyse 2.8.0 and 3.0.0-alpha1 jdiff reports and fix any issues > -- > > Key: HADOOP-13543 > URL: https://issues.apache.org/jira/browse/HADOOP-13543 > Project: Hadoop Common > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli >Priority: Blocker > > Now that we have fixed JDiff report generation for 2.8.0 and above, we should > analyse them. > For the previous releases, I was applying the jdiff patches myself, and > analysed them offline. It's better to track them here now that the reports > are automatically getting generated. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13803) Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one kerberos login
[ https://issues.apache.org/jira/browse/HADOOP-13803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13803. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 > Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one > kerberos login > > > Key: HADOOP-13803 > URL: https://issues.apache.org/jira/browse/HADOOP-13803 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chris Nauroth > Fix For: 3.0.0-alpha2 > > > HADOOP-13081 was released in 3.0.0-alpha1 and the revert wasn't tracked in a > separate JIRA. Filing this one to correct the changelog/release for alpha2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13803) Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one kerberos login
Andrew Wang created HADOOP-13803: Summary: Revert HADOOP-13081 add the ability to create multiple UGIs/subjects from one kerberos login Key: HADOOP-13803 URL: https://issues.apache.org/jira/browse/HADOOP-13803 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Assignee: Chris Nauroth HADOOP-13081 was released in 3.0.0-alpha1 and the revert wasn't tracked in a separate JIRA. Filing this one to correct the changelog/release for alpha2. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13790) Make qbt script executable
Andrew Wang created HADOOP-13790: Summary: Make qbt script executable Key: HADOOP-13790 URL: https://issues.apache.org/jira/browse/HADOOP-13790 Project: Hadoop Common Issue Type: Improvement Components: scripts Affects Versions: 3.0.0-alpha1, 2.8.0 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Trivial Attachments: HADOOP-13790.001.patch Trivial, the qbt script isn't executable, unlike the other scripts in {{dev-support/bin}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13717) Normalize daemonization behavior of the diskbalancer with balancer and mover
[ https://issues.apache.org/jira/browse/HADOOP-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13717. -- Resolution: Invalid Fix Version/s: (was: 3.0.0-alpha2) Reverted and reclosing, thanks again Anu for catching this. > Normalize daemonization behavior of the diskbalancer with balancer and mover > > > Key: HADOOP-13717 > URL: https://issues.apache.org/jira/browse/HADOOP-13717 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: HADOOP-13717.001.patch > > > Issue found when working with the HDFS balancer. > In {{hadoop_daemon_handler}}, it calls {{hadoop_verify_logdir}} even for the > "default" case which calls {{hadoop_start_daemon}}. {{daemon_outfile}} which > specifies the log location isn't even used here, since the command is being > started in the foreground. > I think we can push the {{hadoop_verify_logdir}} call down into > {{hadoop_start_daemon_wrapper}} instead, which does use the outfile. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13717) Normalize daemonization behavior of the diskbalancer with balancer and mover
[ https://issues.apache.org/jira/browse/HADOOP-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13717: -- > Normalize daemonization behavior of the diskbalancer with balancer and mover > > > Key: HADOOP-13717 > URL: https://issues.apache.org/jira/browse/HADOOP-13717 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13717.001.patch > > > Issue found when working with the HDFS balancer. > In {{hadoop_daemon_handler}}, it calls {{hadoop_verify_logdir}} even for the > "default" case which calls {{hadoop_start_daemon}}. {{daemon_outfile}} which > specifies the log location isn't even used here, since the command is being > started in the foreground. > I think we can push the {{hadoop_verify_logdir}} call down into > {{hadoop_start_daemon_wrapper}} instead, which does use the outfile. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13784) Output javadoc inside the target directory
Andrew Wang created HADOOP-13784: Summary: Output javadoc inside the target directory Key: HADOOP-13784 URL: https://issues.apache.org/jira/browse/HADOOP-13784 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 3.0.0-alpha2 Reporter: Andrew Wang Assignee: Andrew Wang HADOOP-8500 cleaned up the javadoc build, but as a result we now have a few javadoc dirs being created outside target folders: {noformat} hadoop-common-project/hadoop-common/api/ hadoop-hdfs-project/hadoop-hdfs-client/api/ hadoop-hdfs-project/hadoop-hdfs/api/ {noformat} Thanks to [~aw] for finding this issue over on HADOOP-8500. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13696) change hadoop-common dependency scope of jsch to provided.
[ https://issues.apache.org/jira/browse/HADOOP-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13696. -- Resolution: Won't Fix Fix Version/s: (was: 3.0.0-alpha2) I filed and linked HADOOP-13769 for splitting this out, thanks again all. > change hadoop-common dependency scope of jsch to provided. > -- > > Key: HADOOP-13696 > URL: https://issues.apache.org/jira/browse/HADOOP-13696 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.3 >Reporter: Steve Loughran >Assignee: Yuanbo Liu >Priority: Minor > Attachments: HADOOP-13696.001.patch > > > The dependency on jsch in Hadoop common is "compile", so it gets everywhere > downstream. Marking it as "provided" would mean that it would only be needed > by those programs which wanted the SFTP filesystem, and, if they wanted to > use a different jsch version, there'd be no maven problems -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13696) change hadoop-common dependency scope of jsch to provided.
[ https://issues.apache.org/jira/browse/HADOOP-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13696: -- > change hadoop-common dependency scope of jsch to provided. > -- > > Key: HADOOP-13696 > URL: https://issues.apache.org/jira/browse/HADOOP-13696 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.7.3 >Reporter: Steve Loughran >Assignee: Yuanbo Liu >Priority: Minor > Attachments: HADOOP-13696.001.patch > > > The dependency on jsch in Hadoop common is "compile", so it gets everywhere > downstream. Marking it as "provided" would mean that it would only be needed > by those programs which wanted the SFTP filesystem, and, if they wanted to > use a different jsch version, there'd be no maven problems -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13759) Split SFTP FileSystem into its own artifact
Andrew Wang created HADOOP-13759: Summary: Split SFTP FileSystem into its own artifact Key: HADOOP-13759 URL: https://issues.apache.org/jira/browse/HADOOP-13759 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.3 Reporter: Andrew Wang As discussed on HADOOP-13696, if we split the SFTP FileSystem into its own artifact, we can save a jsch dependency in Hadoop Common. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13615) Convert uses of AtomicLong for counter metrics to LongAdder
[ https://issues.apache.org/jira/browse/HADOOP-13615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13615. -- Resolution: Duplicate > Convert uses of AtomicLong for counter metrics to LongAdder > --- > > Key: HADOOP-13615 > URL: https://issues.apache.org/jira/browse/HADOOP-13615 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang > > LongAdder (available in JDK8) can provide much better performance than > AtomicLong since it uses thread locals under the hood. > We should consider switching over our uses of AtomicLong and friends over to > LongAdder. > If we want to target for JDK7, we can also pull in the implementation since > it's pure Java (public domain): > http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166e/LongAdder.java?view=co -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-9280) HADOOP-7101 was never merged from 0.23.x to the 1.x branch
[ https://issues.apache.org/jira/browse/HADOOP-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-9280. - Resolution: Won't Fix Old JIRA for branch-1, resolving. > HADOOP-7101 was never merged from 0.23.x to the 1.x branch > -- > > Key: HADOOP-9280 > URL: https://issues.apache.org/jira/browse/HADOOP-9280 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 1.0.4 >Reporter: Claus Ibsen >Assignee: Suresh Srinivas >Priority: Critical > > See HADOOP-7101 > This code fix went into the 0.23 branch. > But was never merged into the 1.x branch, which causing problems for people > upgrading from 0.23 to 1.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-8190) Eclipse plugin fails to access remote cluster
[ https://issues.apache.org/jira/browse/HADOOP-8190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-8190. - Resolution: Not A Problem This JIRA is 4 years old, I'm going to resolve. Please reopen if you intend to work on it. > Eclipse plugin fails to access remote cluster > - > > Key: HADOOP-8190 > URL: https://issues.apache.org/jira/browse/HADOOP-8190 > Project: Hadoop Common > Issue Type: Bug > Components: contrib/eclipse-plugin >Affects Versions: 0.20.205.0 > Environment: Windows and Linux (all) >Reporter: Ambud Sharma >Priority: Critical > Labels: gsoc2012 > Original Estimate: 12h > Remaining Estimate: 12h > > Eclipse plugin fails to access remote file system. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13724) Fix a few typos in site markdown documents
Andrew Wang created HADOOP-13724: Summary: Fix a few typos in site markdown documents Key: HADOOP-13724 URL: https://issues.apache.org/jira/browse/HADOOP-13724 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 3.0.0-alpha1, 2.8.0 Reporter: Andrew Wang Assignee: Ding Fei Priority: Minor Attachments: HADOOP-13708-4.patch New JIRA for HADOOP-13708 since precommit bot is confused by the combination of PRs and patches. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13717) Shell scripts call hadoop_verify_logdir even when command is not started as daemon
Andrew Wang created HADOOP-13717: Summary: Shell scripts call hadoop_verify_logdir even when command is not started as daemon Key: HADOOP-13717 URL: https://issues.apache.org/jira/browse/HADOOP-13717 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Issue found when working with the HDFS balancer. In {{hadoop_daemon_handler}}, it calls {{hadoop_verify_logdir}} even for the "default" case which calls {{hadoop_start_daemon}}. {{daemon_outfile}} which specifies the log location isn't even used here, since the command is being started in the foreground. I think we can push the {{hadoop_verify_logdir}} call down into {{hadoop_start_daemon_wrapper}} instead, which does use the outfile. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13705) Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize code
Andrew Wang created HADOOP-13705: Summary: Revert HADOOP-13534 Remove unused TrashPolicy#getInstance and initialize code Key: HADOOP-13705 URL: https://issues.apache.org/jira/browse/HADOOP-13705 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Assignee: Andrew Wang Per discussion on HADOOP-13700, I'd like to revert HADOOP-13534. It removes a deprecated API, but the 2.x line does not have a release with the new replacement API. This places a burden on downstream applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13699) Configuration does not substitute multiple references to the same var
[ https://issues.apache.org/jira/browse/HADOOP-13699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13699. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Committed to trunk, thanks for reviewing Xiao! > Configuration does not substitute multiple references to the same var > - > > Key: HADOOP-13699 > URL: https://issues.apache.org/jira/browse/HADOOP-13699 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Critical > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-13699.001.patch > > > Config var loop detection was originally introduced by HADOOP-6871. Due to > cycle detection changes in the trunk patch for HADOOP-11506, resolution for > multiple references to the same variable no longer resolved, e.g. > {noformat} > somekey = "${otherkey} ${otherkey}" > {noformat} > This loop detection business is fragile, expensive, and not in branch-2, so > let's reduce it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13699) Configuration does not substitute multiple references to the same var
Andrew Wang created HADOOP-13699: Summary: Configuration does not substitute multiple references to the same var Key: HADOOP-13699 URL: https://issues.apache.org/jira/browse/HADOOP-13699 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Critical Config var loop detection was originally introduced by HADOOP-6871. Due to cycle detection changes in the trunk patch for HADOOP-11506, resolution for multiple references to the same variable no longer resolved, e.g. {noformat} somekey = "${otherkey} ${otherkey}" {noformat} This loop detection business is fragile, expensive, and not in branch-2, so let's reduce it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13689) Do not attach javadoc and sources jars during non-dist build
Andrew Wang created HADOOP-13689: Summary: Do not attach javadoc and sources jars during non-dist build Key: HADOOP-13689 URL: https://issues.apache.org/jira/browse/HADOOP-13689 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.8.0 Reporter: Andrew Wang Assignee: Andrew Wang Looking at maven output when running with "-Pdist", the source plugin "test-jar" and "jar" goals are invoked twice. This is because it's turned on by both the dist profile and on by default. Outside of the release context, it's not that important to have javadoc and source JARs, so I think we can turn it off by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13688) Stop bundling HTML source code in javadoc JARs
Andrew Wang created HADOOP-13688: Summary: Stop bundling HTML source code in javadoc JARs Key: HADOOP-13688 URL: https://issues.apache.org/jira/browse/HADOOP-13688 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 2.8.0 Reporter: Andrew Wang Assignee: Andrew Wang We generate source code with line numbers for inclusion in the javadoc JARs. Given that there's github and other online viewers, this doesn't seem so useful these days. Disabling the "linkSource" option saves us 40MB for the hadoop-common javadoc jar: {noformat} -rw-r--r-- 1 andrew andrew 98M Oct 5 14:44 hadoop-common-3.0.0-alpha2-SNAPSHOT-javadoc.jar -rw-r--r-- 1 andrew andrew 58M Oct 5 15:00 ./hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-alpha2-SNAPSHOT-javadoc.jar {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12010) Inter-operable between Java RS erasure coder and native RS erasure coder
[ https://issues.apache.org/jira/browse/HADOOP-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12010. -- Resolution: Not A Problem Thanks Kai, let's close it out then. I wasn't sure what "Java coder" here meant, but as long as the native coder and new Java coder are compatible, then we're in a good spot. > Inter-operable between Java RS erasure coder and native RS erasure coder > > > Key: HADOOP-12010 > URL: https://issues.apache.org/jira/browse/HADOOP-12010 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha1 >Reporter: Kai Zheng >Assignee: Kai Zheng > Labels: hdfs-ec-3.0-must-do > > It's natural and desired to support inter-operable between the two > implemented RS raw erasure coders, in other words, data encoded in one coder > (like {{RSRawEncoder}}) can be decoded using the other coder (like > {{NativeRSRawDecoder}}). Without this, raw erasure coder will not be > transparent to HDFS data. As such support isn't trivial and involves > challenge work because the two implementations use different encode/decode > matrix generation algorithms, better to have this separately. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11090) [Umbrella] Support Java 8 in Hadoop
[ https://issues.apache.org/jira/browse/HADOOP-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11090. -- Resolution: Fixed I think we're safe to resolve this JIRA. CDH blacklists a few versions, but certified with JDK8 (based on heavily modified 2.6): https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html#pcm_jdk HDP seems similar (2.7 based): http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.0/bk_Installing_HDP_AMB/content/_jdk_requirements.html We also bumped the required JDK version to JDK8 for 3.0.0-alpha1. If there are additional JDK8 issues, let's follow up with separate JIRAs. Thanks all. > [Umbrella] Support Java 8 in Hadoop > --- > > Key: HADOOP-11090 > URL: https://issues.apache.org/jira/browse/HADOOP-11090 > Project: Hadoop Common > Issue Type: New Feature >Reporter: Mohammad Kamrul Islam >Assignee: Mohammad Kamrul Islam > > Java 8 is coming quickly to various clusters. Making sure Hadoop seamlessly > works with Java 8 is important for the Apache community. > > This JIRA is to track the issues/experiences encountered during Java 8 > migration. If you find a potential bug , please create a separate JIRA either > as a sub-task or linked into this JIRA. > If you find a Hadoop or JVM configuration tuning, you can create a JIRA as > well. Or you can add a comment here. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13632) Daemonization does not check process liveness before renicing
Andrew Wang created HADOOP-13632: Summary: Daemonization does not check process liveness before renicing Key: HADOOP-13632 URL: https://issues.apache.org/jira/browse/HADOOP-13632 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang If you try to daemonize a process that is incorrectly configured, it will die quite quickly. However, the daemonization function will still try to renice it even if it's down, leading to something like this for my namenode: {noformat} -> % bin/hdfs --daemon start namenode ERROR: Cannot set priority of namenode process 12036 {noformat} It'd be more user-friendly instead of this renice error, we said that the process couldn't be started. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13615) Convert uses of AtomicLong for counter metrics to LongAdder
Andrew Wang created HADOOP-13615: Summary: Convert uses of AtomicLong for counter metrics to LongAdder Key: HADOOP-13615 URL: https://issues.apache.org/jira/browse/HADOOP-13615 Project: Hadoop Common Issue Type: Improvement Components: metrics Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang LongAdder (available in JDK8) can provide much better performance than AtomicLong since it uses thread locals under the hood. We should consider switching over our uses of AtomicLong and friends over to LongAdder. If we want to target for JDK7, we can also pull in the implementation since it's pure Java (public domain): http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166e/LongAdder.java?view=co -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-7198) Hadoop defaults for web UI ports often fall smack in the middle of Linux ephemeral port range
[ https://issues.apache.org/jira/browse/HADOOP-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-7198. - Resolution: Duplicate Believe this is handled by HDFS-9427 and related JIRAs. Resolving; any remaining ports in the ephemeral range can be addressed in specific JIRAs. > Hadoop defaults for web UI ports often fall smack in the middle of Linux > ephemeral port range > - > > Key: HADOOP-7198 > URL: https://issues.apache.org/jira/browse/HADOOP-7198 > Project: Hadoop Common > Issue Type: Wish >Reporter: Philip Zeyliger >Priority: Trivial > > It turns out (see http://en.wikipedia.org/wiki/Ephemeral_port and > /proc/sys/net/ipv4/ip_local_port_range) that when you bind to port 0, Linux > chooses an ephemeral port. On my default-ridden Ubuntu Maverick box and on > CentOS 5.5, that range is 32768-61000. So, when HBase binds to 60030 or when > mapReduce binds to 50070, there's a small chance that you'll conflict with, > say, an FTP session, or with some other Hadoop daemon that's had a listening > address configured as :0. > I don't know that there's a practical resolution here, since changing the > defaults seems like an ill-fated effort, but if you have any ephemeral port > use, you can run into this. We've now run into it once. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13575) Website shows incorrect 25 January, 2016 as the release date for 2.7.3
[ https://issues.apache.org/jira/browse/HADOOP-13575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13575. -- Resolution: Done I noticed this independently and fixed already, should be good. > Website shows incorrect 25 January, 2016 as the release date for 2.7.3 > -- > > Key: HADOOP-13575 > URL: https://issues.apache.org/jira/browse/HADOOP-13575 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.7.3 >Reporter: Jacek Laskowski > > http://hadoop.apache.org/releases.html shows 2.7.3 released on 25 January, > 2016 which should rather be 25 August, 2016. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13583) Incorporate checkcompatibility script which runs Java API Compliance Checker
Andrew Wang created HADOOP-13583: Summary: Incorporate checkcompatibility script which runs Java API Compliance Checker Key: HADOOP-13583 URL: https://issues.apache.org/jira/browse/HADOOP-13583 Project: Hadoop Common Issue Type: Improvement Components: scripts Affects Versions: 2.6.4 Reporter: Andrew Wang Assignee: Andrew Wang Based on discussion at YETUS-445, this code can't go there, but it's still very useful for release managers. A similar variant of this script has been used for a while by Apache HBase and Apache Kudu, and IMO JACC output is easier to understand than JDiff. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13286) add a S3A scale test to do gunzip and linecount
[ https://issues.apache.org/jira/browse/HADOOP-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13286: -- > add a S3A scale test to do gunzip and linecount > --- > > Key: HADOOP-13286 > URL: https://issues.apache.org/jira/browse/HADOOP-13286 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13286-branch-2-001.patch > > > the HADOOP-13203 patch proposal showed that there were performance problems > downstream which weren't surfacing in the current scale tests. > Trying to decompress the .gz test file and then go through it with LineReader > models a basic use case: parse a .csv.gz data source. > Add this, with metric printing -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13286) add a S3A scale test to do gunzip and linecount
[ https://issues.apache.org/jira/browse/HADOOP-13286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13286. -- Resolution: Duplicate Fix Version/s: (was: 2.8.0) > add a S3A scale test to do gunzip and linecount > --- > > Key: HADOOP-13286 > URL: https://issues.apache.org/jira/browse/HADOOP-13286 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13286-branch-2-001.patch > > > the HADOOP-13203 patch proposal showed that there were performance problems > downstream which weren't surfacing in the current scale tests. > Trying to decompress the .gz test file and then go through it with LineReader > models a basic use case: parse a .csv.gz data source. > Add this, with metric printing -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12976) s3a toString to be meaningful in logs
[ https://issues.apache.org/jira/browse/HADOOP-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12976. -- Resolution: Duplicate > s3a toString to be meaningful in logs > - > > Key: HADOOP-12976 > URL: https://issues.apache.org/jira/browse/HADOOP-12976 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Trivial > Fix For: 2.8.0 > > > today's toString value is just the object ref; better to include the URL of > the FS > Example: > {code} > Cleaning filesystem org.apache.hadoop.fs.s3a.S3AFileSystem@1f069dc1 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12997) s3a to pass PositionedReadable contract tests, improve readFully perf.
[ https://issues.apache.org/jira/browse/HADOOP-12997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12997. -- Resolution: Duplicate Fix Version/s: (was: 2.8.0) > s3a to pass PositionedReadable contract tests, improve readFully perf. > -- > > Key: HADOOP-12997 > URL: https://issues.apache.org/jira/browse/HADOOP-12997 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > > Fix s3a so that it passes the new tests in HADOOP-12994 > Also: optimise readFully so that instead of a sequence of seek-read-seek > operations, it does an opening seek and retains that position as it loops > through the data -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12997) s3a to pass PositionedReadable contract tests, improve readFully perf.
[ https://issues.apache.org/jira/browse/HADOOP-12997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-12997: -- > s3a to pass PositionedReadable contract tests, improve readFully perf. > -- > > Key: HADOOP-12997 > URL: https://issues.apache.org/jira/browse/HADOOP-12997 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > > Fix s3a so that it passes the new tests in HADOOP-12994 > Also: optimise readFully so that instead of a sequence of seek-read-seek > operations, it does an opening seek and retains that position as it loops > through the data -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12976) s3a toString to be meaningful in logs
[ https://issues.apache.org/jira/browse/HADOOP-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-12976: -- > s3a toString to be meaningful in logs > - > > Key: HADOOP-12976 > URL: https://issues.apache.org/jira/browse/HADOOP-12976 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Trivial > Fix For: 2.8.0 > > > today's toString value is just the object ref; better to include the URL of > the FS > Example: > {code} > Cleaning filesystem org.apache.hadoop.fs.s3a.S3AFileSystem@1f069dc1 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12762) task: null java.lang.unsupportedoperationexception: this is supposed to be overridden by subclasses.
[ https://issues.apache.org/jira/browse/HADOOP-12762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12762. -- Resolution: Cannot Reproduce > task: null java.lang.unsupportedoperationexception: this is supposed to be > overridden by subclasses. > > > Key: HADOOP-12762 > URL: https://issues.apache.org/jira/browse/HADOOP-12762 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.2 >Reporter: Padma > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12420) While trying to access Amazon S3 through hadoop-aws(Spark basically) I was getting Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.tr
[ https://issues.apache.org/jira/browse/HADOOP-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-12420: -- > While trying to access Amazon S3 through hadoop-aws(Spark basically) I was > getting Exception in thread "main" java.lang.NoSuchMethodError: > com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V > -- > > Key: HADOOP-12420 > URL: https://issues.apache.org/jira/browse/HADOOP-12420 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.7.1 >Reporter: Tariq Mohammad >Assignee: Tariq Mohammad >Priority: Minor > > While trying to access data stored in Amazon S3 through Apache Spark, which > internally uses hadoop-aws jar I was getting the following exception : > Exception in thread "main" java.lang.NoSuchMethodError: > com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V > Probable reason could be the fact that aws java sdk expects a long parameter > for the setMultipartUploadThreshold(long multiPartThreshold) method, but > hadoop-aws was using a parameter of type int(multiPartThreshold). > I tried using the downloaded hadoop-aws jar and the build through its maven > dependency, but in both the cases I encountered the same exception. Although > I can see private long multiPartThreshold; in hadoop-aws GitHub repo, it's > not getting reflected in the downloaded jar or in the jar created from maven > dependency. > Following lines in the S3AFileSystem class create this difference : > Build from trunk : > private long multiPartThreshold; > this.multiPartThreshold = conf.getLong("fs.s3a.multipart.threshold", > 2147483647L); => Line 267 > Build through maven dependency : > private int multiPartThreshold; > multiPartThreshold = conf.getInt(MIN_MULTIPART_THRESHOLD, > DEFAULT_MIN_MULTIPART_THRESHOLD); => Line 249 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12420) While trying to access Amazon S3 through hadoop-aws(Spark basically) I was getting Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.tr
[ https://issues.apache.org/jira/browse/HADOOP-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12420. -- Resolution: Duplicate Fix Version/s: (was: 2.8.0) > While trying to access Amazon S3 through hadoop-aws(Spark basically) I was > getting Exception in thread "main" java.lang.NoSuchMethodError: > com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V > -- > > Key: HADOOP-12420 > URL: https://issues.apache.org/jira/browse/HADOOP-12420 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.7.1 >Reporter: Tariq Mohammad >Assignee: Tariq Mohammad >Priority: Minor > > While trying to access data stored in Amazon S3 through Apache Spark, which > internally uses hadoop-aws jar I was getting the following exception : > Exception in thread "main" java.lang.NoSuchMethodError: > com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V > Probable reason could be the fact that aws java sdk expects a long parameter > for the setMultipartUploadThreshold(long multiPartThreshold) method, but > hadoop-aws was using a parameter of type int(multiPartThreshold). > I tried using the downloaded hadoop-aws jar and the build through its maven > dependency, but in both the cases I encountered the same exception. Although > I can see private long multiPartThreshold; in hadoop-aws GitHub repo, it's > not getting reflected in the downloaded jar or in the jar created from maven > dependency. > Following lines in the S3AFileSystem class create this difference : > Build from trunk : > private long multiPartThreshold; > this.multiPartThreshold = conf.getLong("fs.s3a.multipart.threshold", > 2147483647L); => Line 267 > Build through maven dependency : > private int multiPartThreshold; > multiPartThreshold = conf.getInt(MIN_MULTIPART_THRESHOLD, > DEFAULT_MIN_MULTIPART_THRESHOLD); => Line 249 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12319) S3AFastOutputStream has no ability to apply backpressure
[ https://issues.apache.org/jira/browse/HADOOP-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-12319: -- > S3AFastOutputStream has no ability to apply backpressure > > > Key: HADOOP-12319 > URL: https://issues.apache.org/jira/browse/HADOOP-12319 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.7.0 >Reporter: Colin Marc >Priority: Critical > > Currently, users of S3AFastOutputStream can control memory usage with a few > settings: {{fs.s3a.threads.core,max}}, which control the number of active > uploads (specifically as arguments to a {{ThreadPoolExecutor}}), and > {{fs.s3a.max.total.tasks}}, which controls the size of the feeding queue for > the {{ThreadPoolExecutor}}. > However, a user can get an almost *guaranteed* crash if the throughput of the > writing job is higher than the total S3 throughput, because there is never > any backpressure or blocking on calls to {{write}}. > If {{fs.s3a.max.total.tasks}} is set high (the default is 1000), then > {{write}} calls will continue to add data to the queue, which can eventually > OOM. But if the user tries to set it lower, then writes will fail when the > queue is full; the {{ThreadPoolExecutor}} will reject the part with > {{java.util.concurrent.RejectedExecutionException}}. > Ideally, calls to {{write}} should *block, not fail* when the queue is full, > so as to apply backpressure on whatever the writing process is. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12319) S3AFastOutputStream has no ability to apply backpressure
[ https://issues.apache.org/jira/browse/HADOOP-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12319. -- Resolution: Duplicate Fix Version/s: (was: 2.8.0) > S3AFastOutputStream has no ability to apply backpressure > > > Key: HADOOP-12319 > URL: https://issues.apache.org/jira/browse/HADOOP-12319 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 2.7.0 >Reporter: Colin Marc >Priority: Critical > > Currently, users of S3AFastOutputStream can control memory usage with a few > settings: {{fs.s3a.threads.core,max}}, which control the number of active > uploads (specifically as arguments to a {{ThreadPoolExecutor}}), and > {{fs.s3a.max.total.tasks}}, which controls the size of the feeding queue for > the {{ThreadPoolExecutor}}. > However, a user can get an almost *guaranteed* crash if the throughput of the > writing job is higher than the total S3 throughput, because there is never > any backpressure or blocking on calls to {{write}}. > If {{fs.s3a.max.total.tasks}} is set high (the default is 1000), then > {{write}} calls will continue to add data to the queue, which can eventually > OOM. But if the user tries to set it lower, then writes will fail when the > queue is full; the {{ThreadPoolExecutor}} will reject the part with > {{java.util.concurrent.RejectedExecutionException}}. > Ideally, calls to {{write}} should *block, not fail* when the queue is full, > so as to apply backpressure on whatever the writing process is. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11874) s3a can throw spurious IOEs on close()
[ https://issues.apache.org/jira/browse/HADOOP-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11874. -- Resolution: Duplicate > s3a can throw spurious IOEs on close() > -- > > Key: HADOOP-11874 > URL: https://issues.apache.org/jira/browse/HADOOP-11874 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 2.8.0 > > > from a code review, it's clear that the issue seen in HADOOP-11851 can > surface in S3a, though with HADOOP-11570, it's less likely. It will only > happen on those cases when abort() isn't called. > The "clean" close() code path needs to catch IOEs from the wrappedStream and > call abort() in that situation too. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-11874) s3a can throw spurious IOEs on close()
[ https://issues.apache.org/jira/browse/HADOOP-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-11874: -- > s3a can throw spurious IOEs on close() > -- > > Key: HADOOP-11874 > URL: https://issues.apache.org/jira/browse/HADOOP-11874 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 2.8.0 > > > from a code review, it's clear that the issue seen in HADOOP-11851 can > surface in S3a, though with HADOOP-11570, it's less likely. It will only > happen on those cases when abort() isn't called. > The "clean" close() code path needs to catch IOEs from the wrappedStream and > call abort() in that situation too. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13409) Andrew's test JIRA
Andrew Wang created HADOOP-13409: Summary: Andrew's test JIRA Key: HADOOP-13409 URL: https://issues.apache.org/jira/browse/HADOOP-13409 Project: Hadoop Common Issue Type: Bug Reporter: Andrew Wang Assignee: Andrew Wang Test JIRA for JIRA interaction script -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13383) Update release notes for 3.0.0-alpha1
[ https://issues.apache.org/jira/browse/HADOOP-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13383. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Thanks for the reviews Akira and Sangjin, committed to trunk and branch-3.0.0-alpha1. > Update release notes for 3.0.0-alpha1 > - > > Key: HADOOP-13383 > URL: https://issues.apache.org/jira/browse/HADOOP-13383 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13383.001.patch, HADOOP-13383.002.patch, > HADOOP-13383.003.patch, HADOOP-13383.004.patch > > > Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), > we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the > right versions, new features and big improvements. > I can put together some notes for HADOOP and HDFS, depending on others for > YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13383) Update release notes for 3.0.0-alpha1
Andrew Wang created HADOOP-13383: Summary: Update release notes for 3.0.0-alpha1 Key: HADOOP-13383 URL: https://issues.apache.org/jira/browse/HADOOP-13383 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Assignee: Andrew Wang Priority: Blocker Per the release instructions (https://wiki.apache.org/hadoop/HowToRelease), we need to update hadoop-project/src/site/markdown/index.md.vm to reflect the right versions, new features and big improvements. I can put together some notes for HADOOP and HDFS, depending on others for YARN and MR. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13348) delete spurious 'master' branch
[ https://issues.apache.org/jira/browse/HADOOP-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13348. -- Resolution: Fixed Assignee: Andrew Wang With the help of INFRA-12223, the master branch has been deleted. Thanks Sean for reporting this issue! > delete spurious 'master' branch > --- > > Key: HADOOP-13348 > URL: https://issues.apache.org/jira/browse/HADOOP-13348 > Project: Hadoop Common > Issue Type: Task > Components: build >Reporter: Sean Busbey >Assignee: Andrew Wang > > Right now the git repo has a branch named 'master' in addition to our 'trunk' > branch. Since 'master' is the common-place name of the 'most recent' branch > in git repositories, this is misleading to new folks. > It looks like the branch is from ~11 months ago. We should remove it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11862) Add support key replicas mechanism for KMS HA
[ https://issues.apache.org/jira/browse/HADOOP-11862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11862. -- Resolution: Not A Problem Resolving, since a full HA story for the KMS also requires a HA backing key provider. Thanks for the nice responses Arun! > Add support key replicas mechanism for KMS HA > - > > Key: HADOOP-11862 > URL: https://issues.apache.org/jira/browse/HADOOP-11862 > Project: Hadoop Common > Issue Type: Improvement > Components: kms >Affects Versions: 2.6.0 >Reporter: dengxiumao > Labels: kms > > The patch [HADOOP-11620|https://issues.apache.org/jira/browse/HADOOP-11620] > only supports specification of multiple hostnames in the kms key provider > uri. it means that it support config as: > {quote} > > hadoop.security.key.provider.path > kms://http@[HOSTNAME1];[HOSTNAME2]:16000/kms > > {quote} > but HA is still not available, if one of KMS instances goes down, Encrypted > files, which encrypted by the keys in the KMS, can not be read. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13303) Detail Informations of KMS High Avalibale
[ https://issues.apache.org/jira/browse/HADOOP-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13303. -- Resolution: Invalid Please use the user list for questions like this, JIRA is for tracking product defects and code changes. Thanks! > Detail Informations of KMS High Avalibale > - > > Key: HADOOP-13303 > URL: https://issues.apache.org/jira/browse/HADOOP-13303 > Project: Hadoop Common > Issue Type: Improvement > Components: ha, kms >Affects Versions: 2.7.2 >Reporter: qiushi fan > > I have some confusions of kms HA recently. > 1. we can set up multiple KMS instances behind a load balancer. Among all > these kms instances, there is only one master kms, others are slave kms. The > master kms can handle Key create/store/rollover/delete operations by directly > contacting with JCE keystore file. The slave kms can handle Key > create/store/rollover/delete operations by delegating it to the master kms. > so although we set up multiple kms, there is only one JCE keystore file, and > only the master kms can access to this file. Both the JCE keystore file and > the master kms don't have a backup. If one of them died, there is no way to > avoid losing data. > Is all of the above true? KMS doesn't have a solution to handle the failure > of master kms and JCE keystore file? > 2. I heard another way to achieve kms HA: make use of > LoadBalancingKMSClientProvider. But I can't find detail informations of > LoadBalancingKMSClientProvider. So why the LoadBalancingKMSClientProvider > can achieve kms HA? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13228) Add delegation token to the connection in DelegationTokenAuthenticator
[ https://issues.apache.org/jira/browse/HADOOP-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13228. -- Resolution: Invalid Reverted from trunk, branch-2, branch-2.8. > Add delegation token to the connection in DelegationTokenAuthenticator > -- > > Key: HADOOP-13228 > URL: https://issues.apache.org/jira/browse/HADOOP-13228 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HADOOP-13228.01.patch, HADOOP-13228.02.patch, > HADOOP-13228.03.patch > > > Following [a comment from another > jira|https://issues.apache.org/jira/browse/HADOOP-13155?focusedCommentId=15308715&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15308715], > create this to specifically handle the delegation token renewal/cancellation > bug in {{DelegationTokenAuthenticatedURL}} and > {{DelegationTokenAuthenticator}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13228) Add delegation token to the connection in DelegationTokenAuthenticator
[ https://issues.apache.org/jira/browse/HADOOP-13228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13228: -- Reopening to revert per comment on HADOOP-13251 > Add delegation token to the connection in DelegationTokenAuthenticator > -- > > Key: HADOOP-13228 > URL: https://issues.apache.org/jira/browse/HADOOP-13228 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HADOOP-13228.01.patch, HADOOP-13228.02.patch, > HADOOP-13228.03.patch > > > Following [a comment from another > jira|https://issues.apache.org/jira/browse/HADOOP-13155?focusedCommentId=15308715&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15308715], > create this to specifically handle the delegation token renewal/cancellation > bug in {{DelegationTokenAuthenticatedURL}} and > {{DelegationTokenAuthenticator}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13175) Remove hadoop-ant from hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13175. -- Resolution: Fixed Hadoop Flags: Incompatible change Fix Version/s: 3.0.0-alpha1 Release Note: The hadoop-ant module in hadoop-tools has been removed. Committed to trunk, thanks Chris for the patch, Jason and Akira for reviewing! > Remove hadoop-ant from hadoop-tools > --- > > Key: HADOOP-13175 > URL: https://issues.apache.org/jira/browse/HADOOP-13175 > Project: Hadoop Common > Issue Type: Task >Reporter: Chris Douglas >Assignee: Chris Douglas > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-13175.001.patch > > > The hadoop-ant code is an ancient kludge unlikely to have any users, still. > We can delete it from trunk as a "scream test" for 3.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-13175) Remove hadoop-ant from hadoop-tools
[ https://issues.apache.org/jira/browse/HADOOP-13175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13175: -- Assignee: Chris Douglas Reopening based on Jason's latest comment. FWIW patch LGTM +1. Chris, anything else we need to do before checking this in for 3.0? > Remove hadoop-ant from hadoop-tools > --- > > Key: HADOOP-13175 > URL: https://issues.apache.org/jira/browse/HADOOP-13175 > Project: Hadoop Common > Issue Type: Task >Reporter: Chris Douglas >Assignee: Chris Douglas > Attachments: HADOOP-13175.001.patch > > > The hadoop-ant code is an ancient kludge unlikely to have any users, still. > We can delete it from trunk as a "scream test" for 3.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-11858) [JDK8] Set minimum version of Hadoop 3 to JDK 8
[ https://issues.apache.org/jira/browse/HADOOP-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-11858. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Target Version/s: (was: ) Committed to trunk. Thanks Robert for the patch and everyone for reviewing! > [JDK8] Set minimum version of Hadoop 3 to JDK 8 > --- > > Key: HADOOP-11858 > URL: https://issues.apache.org/jira/browse/HADOOP-11858 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 3.0.0-alpha1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 3.0.0-alpha1 > > Attachments: HADOOP-11858.001.patch, HADOOP-11858.002.patch, > HADOOP-11858.003.patch > > > Set minimum version of trunk to JDK 8 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-12868) Fix hadoop-openstack undeclared and unused dependencies
[ https://issues.apache.org/jira/browse/HADOOP-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-12868. -- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: (was: ) Committed to trunk, branch-2, branch-2.8. Thanks for the patch [~iwasakims]! > Fix hadoop-openstack undeclared and unused dependencies > --- > > Key: HADOOP-12868 > URL: https://issues.apache.org/jira/browse/HADOOP-12868 > Project: Hadoop Common > Issue Type: Improvement > Components: tools >Affects Versions: 3.0.0-alpha1 >Reporter: Allen Wittenauer >Assignee: Masatake Iwasaki > Fix For: 2.8.0 > > Attachments: HADOOP-12868.001.patch > > > Attempting to compile openstack on a fairly fresh maven repo fails due to > commons-httpclient not being a declared dependency. After that is fixed, > doing a maven dependency:analyze shows other problems. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org