Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/39/ [Aug 27, 2020 10:21:20 PM] (noreply) HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] fate of branch-2.9
+1 on EOL of branch-2.9. Since there are a lot of backported fixes in branch-2.10 after 2.10.0 (2019 Oct 29), it would be nice to release 2.10.1. I'm happy to help release work. Masatake Iwasaki On 2020/08/28 3:25, Mingliang Liu wrote: are there any Hadoop branch-2 releases planned, ever? If so I'll need to backport my s3a directory compatibility patch to whatever is still live. The branch-2 is gone. I think you mean branch-2.10, Steve. Many HBase users are still using Hadoop 2, so I hope Hadoop 2.10.x should still be released at least every 12 months. If there is no volunteer for 2.10.1 RM, I can see how I can help. Thanks, On Thu, Aug 27, 2020 at 8:55 AM John Zhuge wrote: +1 On Thu, Aug 27, 2020 at 6:01 AM Ayush Saxena wrote: +1 -Ayush On 27-Aug-2020, at 6:24 PM, Steve Loughran wrote: +1 are there any Hadoop branch-2 releases planned, ever? If so I'll need to backport my s3a directory compatibility patch to whatever is still live. On Thu, 27 Aug 2020 at 06:55, Wei-Chiu Chuang wrote: Bump up this thread after 6 months. Is anyone still interested in the 2.9 release line? Or are we good to start the EOL process? The 2.9.2 was released in Nov 2018. I'd really like to see the community to converge to fewer release lines and make more frequent releases in each line. Thanks, Weichiu On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang wrote: I think that's a great suggestion. Currently, we make 1 minor release per year, and within each minor release we bring up 1 thousand to 2 thousand commits in it compared with the previous one. I can totally understand it is a big bite for users to swallow. Having a more frequent release cycle, plus LTS and non-LTS releases should help with this. (Of course we will need to make the release preparation much easier, which is currently a pain) I am happy to discuss the release model further in the dev ML. LTS v.s. non-LTS is one suggestion. Another similar issue: In the past Hadoop strived to maintain compatibility. However, this is no longer sustainable as more CVEs coming from our dependencies: netty, jetty, jackson ... etc. In many cases, updating the dependencies brings breaking changes. More recently, especially in Hadoop 3.x, I started to make the effort to update dependencies much more frequently. How do users feel about this change? On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak Maybe Hadoop will benefit from adopting a similar release and support strategy as Java? I.e. designate some releases as LTS and support them for 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other non-LTS releases will be supported for 6 months (or until next release). This should allow to reduce maintenance cost of non-LTS release and provide conservative users desired stability by allowing them to wait for new LTS release and upgrading to it. On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < rupert.mazzu...@gmail.com> wrote: After recently jumping from 2.7.7 to 2.10 without issue myself, I vote for keeping only the 2.10 line. It would seem all other 2.x branches can upgrade to a 2.10.x easily if they feel like upgrading at all, unlike a jump to 3.x, which may require more planning. I also vote for having only one main 3.x branch. Why are there 3.1.x and 3.2.x seemingly competing, and now 3.3.x? For a community that does not have the resources to manage multiple release lines, you guys sure like to multiply release lines a lot. Cheers Rupert Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang : Forwarding the discussion thread from the dev mailing lists to the user mailing lists. I'd like to get an idea of how many users are still on Hadoop 2.9. Please share your thoughts. On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi wrote: +1 Sent from Yahoo Mail on Android On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang< weic...@apache.org> wrote: Hi, Following the discussion to end branch-2.8, I want to start a discussion around what's next with branch-2.9. I am hesitant to use the word "end of life" but consider these facts: * 2.9.0 was released Dec 17, 2017. * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is more than 15 months ago. * no one seems to be interested in being the release manager for 2.9.3. * Most if not all of the active Hadoop contributors are using Hadoop 2.10 or Hadoop 3.x. * We as a community do not have the cycle to manage multiple release line, especially since Hadoop 3.3.0 is coming out soon. It is perhaps the time to gradually reduce our footprint in Hadoop 2.x, and encourage people to upgrade to Hadoop 3.x Thoughts? -- John Zhuge - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17159) Make UGI support forceful relogin from keytab ignoring the last login time
[ https://issues.apache.org/jira/browse/HADOOP-17159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu resolved HADOOP-17159. Fix Version/s: 2.10.1 Hadoop Flags: Reviewed Resolution: Fixed Committed to 2.10.1 and 3.1.5+ see "Fix Version/s". Thank you for your contribution, [~sandeep.guggilam] > Make UGI support forceful relogin from keytab ignoring the last login time > -- > > Key: HADOOP-17159 > URL: https://issues.apache.org/jira/browse/HADOOP-17159 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.3 >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > Fix For: 3.2.2, 2.10.1, 3.3.1, 3.4.0, 3.1.5 > > Time Spent: 40m > Remaining Estimate: 0h > > Currently we have a relogin() method in UGI which attempts to login if there > is no login attempted in the last 10 minutes or configured amount of time > We should also have provision for doing a forceful relogin irrespective of > the time window that the client can choose to use it if needed . Consider the > below scenario: > # SASL Server is reimaged and new keytabs are fetched with refreshing the > password > # SASL client connection to the server would fail when it tries with the > cached service ticket > # We should try to logout to clear the service tickets in cache and then try > to login back in such scenarios. But since the current relogin() doesn't > guarantee a login, it could cause an issue > # A forceful relogin in this case would help after logout > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] fate of branch-2.9
> are there any Hadoop branch-2 releases planned, ever? If so I'll need to backport my s3a directory compatibility patch to whatever is still live. The branch-2 is gone. I think you mean branch-2.10, Steve. Many HBase users are still using Hadoop 2, so I hope Hadoop 2.10.x should still be released at least every 12 months. If there is no volunteer for 2.10.1 RM, I can see how I can help. Thanks, On Thu, Aug 27, 2020 at 8:55 AM John Zhuge wrote: > +1 > > On Thu, Aug 27, 2020 at 6:01 AM Ayush Saxena wrote: > > > +1 > > > > -Ayush > > > > > On 27-Aug-2020, at 6:24 PM, Steve Loughran > > > wrote: > > > > > > > > > > > > +1 > > > > > > are there any Hadoop branch-2 releases planned, ever? If so I'll need > to > > backport my s3a directory compatibility patch to whatever is still live. > > > > > > > > >> On Thu, 27 Aug 2020 at 06:55, Wei-Chiu Chuang > > wrote: > > >> Bump up this thread after 6 months. > > >> > > >> Is anyone still interested in the 2.9 release line? Or are we good to > > start > > >> the EOL process? The 2.9.2 was released in Nov 2018. > > >> > > >> I'd really like to see the community to converge to fewer release > lines > > and > > >> make more frequent releases in each line. > > >> > > >> Thanks, > > >> Weichiu > > >> > > >> > > >> On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang > > wrote: > > >> > > >> > I think that's a great suggestion. > > >> > Currently, we make 1 minor release per year, and within each minor > > release > > >> > we bring up 1 thousand to 2 thousand commits in it compared with the > > >> > previous one. > > >> > I can totally understand it is a big bite for users to swallow. > > Having a > > >> > more frequent release cycle, plus LTS and non-LTS releases should > > help with > > >> > this. (Of course we will need to make the release preparation much > > easier, > > >> > which is currently a pain) > > >> > > > >> > I am happy to discuss the release model further in the dev ML. LTS > > v.s. > > >> > non-LTS is one suggestion. > > >> > > > >> > Another similar issue: In the past Hadoop strived to > > >> > maintain compatibility. However, this is no longer sustainable as > > more CVEs > > >> > coming from our dependencies: netty, jetty, jackson ... etc. > > >> > In many cases, updating the dependencies brings breaking changes. > More > > >> > recently, especially in Hadoop 3.x, I started to make the effort to > > update > > >> > dependencies much more frequently. How do users feel about this > > change? > > >> > > > >> > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak > > > >> > wrote: > > >> > > > >> >> Maybe Hadoop will benefit from adopting a similar release and > support > > >> >> strategy as Java? I.e. designate some releases as LTS and support > > them for > > >> >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other > > non-LTS > > >> >> releases will be supported for 6 months (or until next release). > This > > >> >> should allow to reduce maintenance cost of non-LTS release and > > provide > > >> >> conservative users desired stability by allowing them to wait for > > new LTS > > >> >> release and upgrading to it. > > >> >> > > >> >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < > > rupert.mazzu...@gmail.com> > > >> >> wrote: > > >> >> > > >> >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I > > vote > > >> >>> for keeping only the 2.10 line. > > >> >>> It would seem all other 2.x branches can upgrade to a 2.10.x > easily > > if > > >> >>> they feel like upgrading at all, > > >> >>> unlike a jump to 3.x, which may require more planning. > > >> >>> > > >> >>> I also vote for having only one main 3.x branch. Why are there > > 3.1.x and > > >> >>> 3.2.x seemingly competing, > > >> >>> and now 3.3.x? For a community that does not have the resources to > > >> >>> manage multiple release lines, > > >> >>> you guys sure like to multiply release lines a lot. > > >> >>> > > >> >>> Cheers > > >> >>> Rupert > > >> >>> > > >> >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang > > >> >>> : > > >> >>> > > >> Forwarding the discussion thread from the dev mailing lists to > the > > user > > >> mailing lists. > > >> > > >> I'd like to get an idea of how many users are still on Hadoop > 2.9. > > >> Please share your thoughts. > > >> > > >> On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi > > >> wrote: > > >> > > >> > +1 > > >> > > > >> > Sent from Yahoo Mail on Android > > >> > > > >> > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang< > > weic...@apache.org> > > >> > wrote: Hi, > > >> > > > >> > Following the discussion to end branch-2.8, I want to start a > > >> > discussion > > >> > around what's next with branch-2.9. I am hesitant to use the > word > > "end > > >> > of > > >> > life" but consider these facts: > > >> > > > >> > * 2.9.0 was released Dec 17, 2017. > > >> > * 2.9.2, the last 2.9.x release, went out Nov 19 2018,
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/ [Aug 26, 2020 7:33:08 AM] (Hemanth Boyina) HDFS-15536. RBF: Clear Quota in Router was not consistent. [Aug 26, 2020 7:38:17 AM] (pjoseph) YARN-1806. Add ThreadDump Option in YARN UI2 to fetch for running containers [Aug 26, 2020 2:15:24 PM] (noreply) HADOOP-17224. Install Intel ISA-L library in Dockerfile. (#2243) [Aug 26, 2020 5:41:10 PM] (Mingliang Liu) Revert "HADOOP-17159 Ability for forceful relogin in UserGroupInformation class (#2197)" -1 overall The following subsystems voted -1: asflicense pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.hdfs.TestFileChecksum hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks hadoop.hdfs.TestFileChecksumCompositeCrc hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader hadoop.hdfs.server.mover.TestMover hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer hadoop.hdfs.TestMaintenanceState hadoop.hdfs.TestErasureCodingExerciseAPIs hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy hadoop.hdfs.server.federation.router.TestRouterMultiRack hadoop.hdfs.server.federation.router.TestRouterAllResolver hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.yarn.service.TestServiceAM hadoop.yarn.sls.appmaster.TestAMSimulator cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-compile-cc-root.txt [48K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-compile-javac-root.txt [568K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-patch-pylint.txt [60K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-patch-shelldocs.txt [44K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/whitespace-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/whitespace-tabs.txt [1.9M] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/diff-javadoc-javadoc-root.txt [1.3M] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [632K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/247/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [64K] https://ci-hadoop.apache.or
Re: [DISCUSS] fate of branch-2.9
+1 On Thu, Aug 27, 2020 at 6:01 AM Ayush Saxena wrote: > +1 > > -Ayush > > > On 27-Aug-2020, at 6:24 PM, Steve Loughran > wrote: > > > > > > > > +1 > > > > are there any Hadoop branch-2 releases planned, ever? If so I'll need to > backport my s3a directory compatibility patch to whatever is still live. > > > > > >> On Thu, 27 Aug 2020 at 06:55, Wei-Chiu Chuang > wrote: > >> Bump up this thread after 6 months. > >> > >> Is anyone still interested in the 2.9 release line? Or are we good to > start > >> the EOL process? The 2.9.2 was released in Nov 2018. > >> > >> I'd really like to see the community to converge to fewer release lines > and > >> make more frequent releases in each line. > >> > >> Thanks, > >> Weichiu > >> > >> > >> On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang > wrote: > >> > >> > I think that's a great suggestion. > >> > Currently, we make 1 minor release per year, and within each minor > release > >> > we bring up 1 thousand to 2 thousand commits in it compared with the > >> > previous one. > >> > I can totally understand it is a big bite for users to swallow. > Having a > >> > more frequent release cycle, plus LTS and non-LTS releases should > help with > >> > this. (Of course we will need to make the release preparation much > easier, > >> > which is currently a pain) > >> > > >> > I am happy to discuss the release model further in the dev ML. LTS > v.s. > >> > non-LTS is one suggestion. > >> > > >> > Another similar issue: In the past Hadoop strived to > >> > maintain compatibility. However, this is no longer sustainable as > more CVEs > >> > coming from our dependencies: netty, jetty, jackson ... etc. > >> > In many cases, updating the dependencies brings breaking changes. More > >> > recently, especially in Hadoop 3.x, I started to make the effort to > update > >> > dependencies much more frequently. How do users feel about this > change? > >> > > >> > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak > >> > wrote: > >> > > >> >> Maybe Hadoop will benefit from adopting a similar release and support > >> >> strategy as Java? I.e. designate some releases as LTS and support > them for > >> >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other > non-LTS > >> >> releases will be supported for 6 months (or until next release). This > >> >> should allow to reduce maintenance cost of non-LTS release and > provide > >> >> conservative users desired stability by allowing them to wait for > new LTS > >> >> release and upgrading to it. > >> >> > >> >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < > rupert.mazzu...@gmail.com> > >> >> wrote: > >> >> > >> >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I > vote > >> >>> for keeping only the 2.10 line. > >> >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily > if > >> >>> they feel like upgrading at all, > >> >>> unlike a jump to 3.x, which may require more planning. > >> >>> > >> >>> I also vote for having only one main 3.x branch. Why are there > 3.1.x and > >> >>> 3.2.x seemingly competing, > >> >>> and now 3.3.x? For a community that does not have the resources to > >> >>> manage multiple release lines, > >> >>> you guys sure like to multiply release lines a lot. > >> >>> > >> >>> Cheers > >> >>> Rupert > >> >>> > >> >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang > >> >>> : > >> >>> > >> Forwarding the discussion thread from the dev mailing lists to the > user > >> mailing lists. > >> > >> I'd like to get an idea of how many users are still on Hadoop 2.9. > >> Please share your thoughts. > >> > >> On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi > >> wrote: > >> > >> > +1 > >> > > >> > Sent from Yahoo Mail on Android > >> > > >> > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang< > weic...@apache.org> > >> > wrote: Hi, > >> > > >> > Following the discussion to end branch-2.8, I want to start a > >> > discussion > >> > around what's next with branch-2.9. I am hesitant to use the word > "end > >> > of > >> > life" but consider these facts: > >> > > >> > * 2.9.0 was released Dec 17, 2017. > >> > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is > more > >> > than > >> > 15 months ago. > >> > * no one seems to be interested in being the release manager for > 2.9.3. > >> > * Most if not all of the active Hadoop contributors are using > Hadoop > >> > 2.10 > >> > or Hadoop 3.x. > >> > * We as a community do not have the cycle to manage multiple > release > >> > line, > >> > especially since Hadoop 3.3.0 is coming out soon. > >> > > >> > It is perhaps the time to gradually reduce our footprint in Hadoop > >> > 2.x, and > >> > encourage people to upgrade to Hadoop 3.x > >> > > >> > Thoughts? > >> > > >> > > -- John Zhuge
[jira] [Created] (HADOOP-17234) Add .asf.yaml to allow github and jira integration
Ayush Saxena created HADOOP-17234: - Summary: Add .asf.yaml to allow github and jira integration Key: HADOOP-17234 URL: https://issues.apache.org/jira/browse/HADOOP-17234 Project: Hadoop Common Issue Type: Task Reporter: Ayush Saxena Assignee: Ayush Saxena As of now the default for github is set only to worklog, To enable link and label, We need to add this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [Virtual MEETUP]: Migration to Hadoop 3
Wei-Chiu, our plans are tentative at the moment, but we have internally discussed migrating from 2.10 to 3.2. And, thank you all again for the great participation, especially those of you in timezones where it was late in the evening. -Eric On Thursday, August 27, 2020, 12:47:43 AM CDT, Wei-Chiu Chuang wrote: Thanks Brahma, Eric, do you have a target Hadoop 3 release line in mind? The "unofficial" plan here at Cloudera is to rebase our current dev codebase from Hadoop 3.1.1 to 3.3 some time later. The Hadoop 3.1 code line will approach its 3rd anniversary by this year's end so perhaps we can start to sunset it. On Wed, Aug 26, 2020 at 10:51 AM Brahma Reddy Battula wrote: > One more update from me. > > We didn't face any issues with YARN, for HDFS you can have a look at the > following jira's. > > https://issues.apache.org/jira/browse/HDFS-13596 > https://issues.apache.org/jira/browse/HDFS-14396 > https://issues.apache.org/jira/browse/HDFS-14509 > > Following jira is incompatible for ACL commands.Only hadoop-3 clients will > work against hadoop-3 server during the upgrade. > > https://issues.apache.org/jira/browse/HDFS-6984 > > > > On Wed, Aug 26, 2020 at 11:06 PM Brahma Reddy Battula > wrote: > > > > > Hi Eric, > > > > check the following references for the same. > > > > 01/02/2020 Didi talked about their large scale HDFS cluster upgrade > > experience. > > > > Slides: > > https://drive.google.com/open?id=1iwJ1asalYfgnOCBuE-RfeG-NpSocjIcy > > > > Recording: > > > https://cloudera.zoom.us/rec/share/7MF_dLX0339OY5391xvkZP8NLrXieaa8gyZK-fYJnUkGOUUXvaUh5cl_6AVYetQl > > > > Didi studied two upgrade approaches from the community documentation: > > express upgrade and rolling upgrade. Rolling upgrade was selected. > > > > Yahoo Japan was trying out from hadoop-2.6 to hadop-3.2.1 > > > > https://techblog.yahoo.co.jp/entry/20191206786320/ > > > > On Wed, Aug 26, 2020 at 6:56 PM epa...@apache.org > > wrote: > > > >> Hello. Just a reminder that today I would like to invite you all to > >> discuss your > >> experiences migrating from Hadoop 2 to Hadoop 3. > >> > >> -Eric > >> > >> On Monday, August 24, 2020, 1:58:37 PM CDT, epa...@apache.org < > >> epa...@apache.org> wrote: > >> > >> Hello everyone! > >> > >> We are considering migrating to Hadoop 3, and we would be very > interested > >> to > >> hear about your experiences. If you have migrated from Hadoop 2 to > Hadoop > >> 3 > >> and can provide insights, please kindly consider attending the > following: > >> > >> Date: Wednesday, Aug 26, 2020 > >> Time: 10:00 A.M. PDT / 12:00 P.M. CDT / 01:00 P.M. EDT / 05:00 P.M. GMT > >> Location: Zoom: https://cloudera.zoom.us/j/880548968 > >> > >> Hope to see you there! > >> > >> Thank you! > >> Eric Payne > >> @ Verizon Media > >> > >> - > >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >> > >> > > > > -- > > > > > > > > --Brahma Reddy Battula > > > > > -- > > > > --Brahma Reddy Battula > - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16934) Test failure in TestAbfsOutputStream because of HADOOP-16910
[ https://issues.apache.org/jira/browse/HADOOP-16934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehakmeet Singh resolved HADOOP-16934. -- Resolution: Duplicate > Test failure in TestAbfsOutputStream because of HADOOP-16910 > > > Key: HADOOP-16934 > URL: https://issues.apache.org/jira/browse/HADOOP-16934 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.1 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Labels: release-blocker > > Failure in TestAbfsOutputStream.java due to not passing FileSystem.Statistics > in AbfsOutputstream. > job: > - Passing Statistics through AbfsOutputStream. > - Closing streams in the test -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17065) Adding Network Counters in ABFS
[ https://issues.apache.org/jira/browse/HADOOP-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehakmeet Singh resolved HADOOP-17065. -- Resolution: Fixed > Adding Network Counters in ABFS > --- > > Key: HADOOP-17065 > URL: https://issues.apache.org/jira/browse/HADOOP-17065 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Fix For: 3.3.1 > > > Network Counters to be added in ABFS: > |CONNECTIONS_MADE|Number of times connection was made with Azure Data Lake| > |SEND_REQUESTS|Number of send requests| > |GET_RESPONSE|Number of response gotten| > |BYTES_SEND|Number of bytes send| > |BYTES_RECEIVED|Number of bytes received| > |READ_THROTTLE|Number of times throttled while read operation| > |WRITE_THROTTLE|Number of times throttled while write operation| > propose: > * Adding these counters as part of AbfsStatistic already made in > HADOOP-17016. > * Increment of counters across Abfs Network services. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17129) Validating storage keys in ABFS correctly
[ https://issues.apache.org/jira/browse/HADOOP-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehakmeet Singh resolved HADOOP-17129. -- Fix Version/s: 3.3.1 Resolution: Fixed > Validating storage keys in ABFS correctly > - > > Key: HADOOP-17129 > URL: https://issues.apache.org/jira/browse/HADOOP-17129 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Fix For: 3.3.1 > > > Storage Keys in ABFS should be validated after the keys have been loaded. > work: > - Remove the previous validation of storage keys. > - Validate at the correct place. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] fate of branch-2.9
+1 -Ayush > On 27-Aug-2020, at 6:24 PM, Steve Loughran > wrote: > > > > +1 > > are there any Hadoop branch-2 releases planned, ever? If so I'll need to > backport my s3a directory compatibility patch to whatever is still live. > > >> On Thu, 27 Aug 2020 at 06:55, Wei-Chiu Chuang wrote: >> Bump up this thread after 6 months. >> >> Is anyone still interested in the 2.9 release line? Or are we good to start >> the EOL process? The 2.9.2 was released in Nov 2018. >> >> I'd really like to see the community to converge to fewer release lines and >> make more frequent releases in each line. >> >> Thanks, >> Weichiu >> >> >> On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang wrote: >> >> > I think that's a great suggestion. >> > Currently, we make 1 minor release per year, and within each minor release >> > we bring up 1 thousand to 2 thousand commits in it compared with the >> > previous one. >> > I can totally understand it is a big bite for users to swallow. Having a >> > more frequent release cycle, plus LTS and non-LTS releases should help with >> > this. (Of course we will need to make the release preparation much easier, >> > which is currently a pain) >> > >> > I am happy to discuss the release model further in the dev ML. LTS v.s. >> > non-LTS is one suggestion. >> > >> > Another similar issue: In the past Hadoop strived to >> > maintain compatibility. However, this is no longer sustainable as more CVEs >> > coming from our dependencies: netty, jetty, jackson ... etc. >> > In many cases, updating the dependencies brings breaking changes. More >> > recently, especially in Hadoop 3.x, I started to make the effort to update >> > dependencies much more frequently. How do users feel about this change? >> > >> > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak >> > wrote: >> > >> >> Maybe Hadoop will benefit from adopting a similar release and support >> >> strategy as Java? I.e. designate some releases as LTS and support them for >> >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other non-LTS >> >> releases will be supported for 6 months (or until next release). This >> >> should allow to reduce maintenance cost of non-LTS release and provide >> >> conservative users desired stability by allowing them to wait for new LTS >> >> release and upgrading to it. >> >> >> >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco >> >> wrote: >> >> >> >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I vote >> >>> for keeping only the 2.10 line. >> >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily if >> >>> they feel like upgrading at all, >> >>> unlike a jump to 3.x, which may require more planning. >> >>> >> >>> I also vote for having only one main 3.x branch. Why are there 3.1.x and >> >>> 3.2.x seemingly competing, >> >>> and now 3.3.x? For a community that does not have the resources to >> >>> manage multiple release lines, >> >>> you guys sure like to multiply release lines a lot. >> >>> >> >>> Cheers >> >>> Rupert >> >>> >> >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang >> >>> : >> >>> >> Forwarding the discussion thread from the dev mailing lists to the user >> mailing lists. >> >> I'd like to get an idea of how many users are still on Hadoop 2.9. >> Please share your thoughts. >> >> On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi >> wrote: >> >> > +1 >> > >> > Sent from Yahoo Mail on Android >> > >> > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang >> > wrote: Hi, >> > >> > Following the discussion to end branch-2.8, I want to start a >> > discussion >> > around what's next with branch-2.9. I am hesitant to use the word "end >> > of >> > life" but consider these facts: >> > >> > * 2.9.0 was released Dec 17, 2017. >> > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is more >> > than >> > 15 months ago. >> > * no one seems to be interested in being the release manager for 2.9.3. >> > * Most if not all of the active Hadoop contributors are using Hadoop >> > 2.10 >> > or Hadoop 3.x. >> > * We as a community do not have the cycle to manage multiple release >> > line, >> > especially since Hadoop 3.3.0 is coming out soon. >> > >> > It is perhaps the time to gradually reduce our footprint in Hadoop >> > 2.x, and >> > encourage people to upgrade to Hadoop 3.x >> > >> > Thoughts? >> > >> >
Re: [DISCUSS] fate of branch-2.9
+1 are there any Hadoop branch-2 releases planned, ever? If so I'll need to backport my s3a directory compatibility patch to whatever is still live. On Thu, 27 Aug 2020 at 06:55, Wei-Chiu Chuang wrote: > Bump up this thread after 6 months. > > Is anyone still interested in the 2.9 release line? Or are we good to start > the EOL process? The 2.9.2 was released in Nov 2018. > > I'd really like to see the community to converge to fewer release lines and > make more frequent releases in each line. > > Thanks, > Weichiu > > > On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang wrote: > > > I think that's a great suggestion. > > Currently, we make 1 minor release per year, and within each minor > release > > we bring up 1 thousand to 2 thousand commits in it compared with the > > previous one. > > I can totally understand it is a big bite for users to swallow. Having a > > more frequent release cycle, plus LTS and non-LTS releases should help > with > > this. (Of course we will need to make the release preparation much > easier, > > which is currently a pain) > > > > I am happy to discuss the release model further in the dev ML. LTS v.s. > > non-LTS is one suggestion. > > > > Another similar issue: In the past Hadoop strived to > > maintain compatibility. However, this is no longer sustainable as more > CVEs > > coming from our dependencies: netty, jetty, jackson ... etc. > > In many cases, updating the dependencies brings breaking changes. More > > recently, especially in Hadoop 3.x, I started to make the effort to > update > > dependencies much more frequently. How do users feel about this change? > > > > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak > > wrote: > > > >> Maybe Hadoop will benefit from adopting a similar release and support > >> strategy as Java? I.e. designate some releases as LTS and support them > for > >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other non-LTS > >> releases will be supported for 6 months (or until next release). This > >> should allow to reduce maintenance cost of non-LTS release and provide > >> conservative users desired stability by allowing them to wait for new > LTS > >> release and upgrading to it. > >> > >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < > rupert.mazzu...@gmail.com> > >> wrote: > >> > >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I vote > >>> for keeping only the 2.10 line. > >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily if > >>> they feel like upgrading at all, > >>> unlike a jump to 3.x, which may require more planning. > >>> > >>> I also vote for having only one main 3.x branch. Why are there 3.1.x > and > >>> 3.2.x seemingly competing, > >>> and now 3.3.x? For a community that does not have the resources to > >>> manage multiple release lines, > >>> you guys sure like to multiply release lines a lot. > >>> > >>> Cheers > >>> Rupert > >>> > >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang > >>> : > >>> > Forwarding the discussion thread from the dev mailing lists to the > user > mailing lists. > > I'd like to get an idea of how many users are still on Hadoop 2.9. > Please share your thoughts. > > On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi > wrote: > > > +1 > > > > Sent from Yahoo Mail on Android > > > > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang > > > wrote: Hi, > > > > Following the discussion to end branch-2.8, I want to start a > > discussion > > around what's next with branch-2.9. I am hesitant to use the word > "end > > of > > life" but consider these facts: > > > > * 2.9.0 was released Dec 17, 2017. > > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is more > > than > > 15 months ago. > > * no one seems to be interested in being the release manager for > 2.9.3. > > * Most if not all of the active Hadoop contributors are using Hadoop > > 2.10 > > or Hadoop 3.x. > > * We as a community do not have the cycle to manage multiple release > > line, > > especially since Hadoop 3.3.0 is coming out soon. > > > > It is perhaps the time to gradually reduce our footprint in Hadoop > > 2.x, and > > encourage people to upgrade to Hadoop 3.x > > > > Thoughts? > > > > >
Re: [DISCUSS] fate of branch-2.9
+1 Thanks Sunil On Thu, Aug 27, 2020 at 3:49 PM Xiaoqiao He wrote: > +1 for putting 2.9 release lines to EOL. > > Thanks, > Hexiaoqiao > > On Thu, Aug 27, 2020 at 3:14 PM Mingliang Liu wrote: > >> +1 for putting 2.9 lines to EOL. >> >> Let's focus on 2.10 releases for Hadoop 2. Also is there any plan for >> 2.10.1? It has been 11 months since 2.10 first release. >> >> Thanks, >> >> On Wed, Aug 26, 2020 at 10:57 PM Wei-Chiu Chuang >> wrote: >> >> > Bump up this thread after 6 months. >> > >> > Is anyone still interested in the 2.9 release line? Or are we good to >> start >> > the EOL process? The 2.9.2 was released in Nov 2018. >> > >> > I'd really like to see the community to converge to fewer release lines >> and >> > make more frequent releases in each line. >> > >> > Thanks, >> > Weichiu >> > >> > >> > On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang >> wrote: >> > >> > > I think that's a great suggestion. >> > > Currently, we make 1 minor release per year, and within each minor >> > release >> > > we bring up 1 thousand to 2 thousand commits in it compared with the >> > > previous one. >> > > I can totally understand it is a big bite for users to swallow. >> Having a >> > > more frequent release cycle, plus LTS and non-LTS releases should help >> > with >> > > this. (Of course we will need to make the release preparation much >> > easier, >> > > which is currently a pain) >> > > >> > > I am happy to discuss the release model further in the dev ML. LTS >> v.s. >> > > non-LTS is one suggestion. >> > > >> > > Another similar issue: In the past Hadoop strived to >> > > maintain compatibility. However, this is no longer sustainable as more >> > CVEs >> > > coming from our dependencies: netty, jetty, jackson ... etc. >> > > In many cases, updating the dependencies brings breaking changes. More >> > > recently, especially in Hadoop 3.x, I started to make the effort to >> > update >> > > dependencies much more frequently. How do users feel about this >> change? >> > > >> > > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak >> > > wrote: >> > > >> > >> Maybe Hadoop will benefit from adopting a similar release and support >> > >> strategy as Java? I.e. designate some releases as LTS and support >> them >> > for >> > >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other >> non-LTS >> > >> releases will be supported for 6 months (or until next release). This >> > >> should allow to reduce maintenance cost of non-LTS release and >> provide >> > >> conservative users desired stability by allowing them to wait for new >> > LTS >> > >> release and upgrading to it. >> > >> >> > >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < >> > rupert.mazzu...@gmail.com> >> > >> wrote: >> > >> >> > >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I >> vote >> > >>> for keeping only the 2.10 line. >> > >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily >> if >> > >>> they feel like upgrading at all, >> > >>> unlike a jump to 3.x, which may require more planning. >> > >>> >> > >>> I also vote for having only one main 3.x branch. Why are there 3.1.x >> > and >> > >>> 3.2.x seemingly competing, >> > >>> and now 3.3.x? For a community that does not have the resources to >> > >>> manage multiple release lines, >> > >>> you guys sure like to multiply release lines a lot. >> > >>> >> > >>> Cheers >> > >>> Rupert >> > >>> >> > >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang >> > >>> : >> > >>> >> > Forwarding the discussion thread from the dev mailing lists to the >> > user >> > mailing lists. >> > >> > I'd like to get an idea of how many users are still on Hadoop 2.9. >> > Please share your thoughts. >> > >> > On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi >> > wrote: >> > >> > > +1 >> > > >> > > Sent from Yahoo Mail on Android >> > > >> > > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang< >> weic...@apache.org >> > > >> > > wrote: Hi, >> > > >> > > Following the discussion to end branch-2.8, I want to start a >> > > discussion >> > > around what's next with branch-2.9. I am hesitant to use the word >> > "end >> > > of >> > > life" but consider these facts: >> > > >> > > * 2.9.0 was released Dec 17, 2017. >> > > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is >> more >> > > than >> > > 15 months ago. >> > > * no one seems to be interested in being the release manager for >> > 2.9.3. >> > > * Most if not all of the active Hadoop contributors are using >> Hadoop >> > > 2.10 >> > > or Hadoop 3.x. >> > > * We as a community do not have the cycle to manage multiple >> release >> > > line, >> > > especially since Hadoop 3.3.0 is coming out soon. >> > > >> > > It is perhaps the time to gradually reduce our footprint in Hadoop >> > > 2.x, and >> > > encourage people to upgrade to Hadoop 3.x >> > > >> >
[jira] [Resolved] (HADOOP-17194) Adding Context class for AbfsClient to pass AbfsConfigurations to limit number of parameters
[ https://issues.apache.org/jira/browse/HADOOP-17194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17194. - Fix Version/s: 3.3.1 Resolution: Fixed > Adding Context class for AbfsClient to pass AbfsConfigurations to limit > number of parameters > - > > Key: HADOOP-17194 > URL: https://issues.apache.org/jira/browse/HADOOP-17194 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Mehakmeet Singh >Assignee: Mehakmeet Singh >Priority: Major > Fix For: 3.3.1 > > > To limit the growing number of parameters in AbfsClient for > AbfsConfigurations, introducing a context class to pass the > AbfsConfigurations from. > This would help in passing the AbfsConfigurations to further classes like > AbfsRestOperation and AbfsHttpOperation also. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] fate of branch-2.9
+1 for putting 2.9 release lines to EOL. Thanks, Hexiaoqiao On Thu, Aug 27, 2020 at 3:14 PM Mingliang Liu wrote: > +1 for putting 2.9 lines to EOL. > > Let's focus on 2.10 releases for Hadoop 2. Also is there any plan for > 2.10.1? It has been 11 months since 2.10 first release. > > Thanks, > > On Wed, Aug 26, 2020 at 10:57 PM Wei-Chiu Chuang > wrote: > > > Bump up this thread after 6 months. > > > > Is anyone still interested in the 2.9 release line? Or are we good to > start > > the EOL process? The 2.9.2 was released in Nov 2018. > > > > I'd really like to see the community to converge to fewer release lines > and > > make more frequent releases in each line. > > > > Thanks, > > Weichiu > > > > > > On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang > wrote: > > > > > I think that's a great suggestion. > > > Currently, we make 1 minor release per year, and within each minor > > release > > > we bring up 1 thousand to 2 thousand commits in it compared with the > > > previous one. > > > I can totally understand it is a big bite for users to swallow. Having > a > > > more frequent release cycle, plus LTS and non-LTS releases should help > > with > > > this. (Of course we will need to make the release preparation much > > easier, > > > which is currently a pain) > > > > > > I am happy to discuss the release model further in the dev ML. LTS v.s. > > > non-LTS is one suggestion. > > > > > > Another similar issue: In the past Hadoop strived to > > > maintain compatibility. However, this is no longer sustainable as more > > CVEs > > > coming from our dependencies: netty, jetty, jackson ... etc. > > > In many cases, updating the dependencies brings breaking changes. More > > > recently, especially in Hadoop 3.x, I started to make the effort to > > update > > > dependencies much more frequently. How do users feel about this change? > > > > > > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak > > > wrote: > > > > > >> Maybe Hadoop will benefit from adopting a similar release and support > > >> strategy as Java? I.e. designate some releases as LTS and support them > > for > > >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other > non-LTS > > >> releases will be supported for 6 months (or until next release). This > > >> should allow to reduce maintenance cost of non-LTS release and provide > > >> conservative users desired stability by allowing them to wait for new > > LTS > > >> release and upgrading to it. > > >> > > >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < > > rupert.mazzu...@gmail.com> > > >> wrote: > > >> > > >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I > vote > > >>> for keeping only the 2.10 line. > > >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily > if > > >>> they feel like upgrading at all, > > >>> unlike a jump to 3.x, which may require more planning. > > >>> > > >>> I also vote for having only one main 3.x branch. Why are there 3.1.x > > and > > >>> 3.2.x seemingly competing, > > >>> and now 3.3.x? For a community that does not have the resources to > > >>> manage multiple release lines, > > >>> you guys sure like to multiply release lines a lot. > > >>> > > >>> Cheers > > >>> Rupert > > >>> > > >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang > > >>> : > > >>> > > Forwarding the discussion thread from the dev mailing lists to the > > user > > mailing lists. > > > > I'd like to get an idea of how many users are still on Hadoop 2.9. > > Please share your thoughts. > > > > On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi > > wrote: > > > > > +1 > > > > > > Sent from Yahoo Mail on Android > > > > > > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang< > weic...@apache.org > > > > > > wrote: Hi, > > > > > > Following the discussion to end branch-2.8, I want to start a > > > discussion > > > around what's next with branch-2.9. I am hesitant to use the word > > "end > > > of > > > life" but consider these facts: > > > > > > * 2.9.0 was released Dec 17, 2017. > > > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is > more > > > than > > > 15 months ago. > > > * no one seems to be interested in being the release manager for > > 2.9.3. > > > * Most if not all of the active Hadoop contributors are using > Hadoop > > > 2.10 > > > or Hadoop 3.x. > > > * We as a community do not have the cycle to manage multiple > release > > > line, > > > especially since Hadoop 3.3.0 is coming out soon. > > > > > > It is perhaps the time to gradually reduce our footprint in Hadoop > > > 2.x, and > > > encourage people to upgrade to Hadoop 3.x > > > > > > Thoughts? > > > > > > > > > > > -- > L >
[jira] [Resolved] (HADOOP-17233) Fix unit test of HDFS-13934
[
https://issues.apache.org/jira/browse/HADOOP-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
fanrui resolved HADOOP-17233.
-
Resolution: Duplicate
> Fix unit test of HDFS-13934
> ---
>
> Key: HADOOP-17233
> URL: https://issues.apache.org/jira/browse/HADOOP-17233
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Reporter: fanrui
>Priority: Major
>
> unit test failed of
> org.apache.hadoop.fs.contract.AbstractContractMultipartUploaderTest#testConcurrentUploads.
> Exception:
> {code:java}
> java.lang.IllegalArgumentExceptionjava.lang.IllegalArgumentException at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:127) at
> org.apache.hadoop.test.LambdaTestUtils$ProportionalRetryInterval.(LambdaTestUtils.java:907)
> at
> org.apache.hadoop.fs.contract.AbstractContractMultipartUploaderTest.testConcurrentUploads(AbstractContractMultipartUploaderTest.java:815){code}
> h3. Reason:
> {code:java}
> public ProportionalRetryInterval(int intervalMillis,
> int maxIntervalMillis) {
> Preconditions.checkArgument(intervalMillis > 0);
> Preconditions.checkArgument(maxIntervalMillis > 0);
> this.intervalMillis = intervalMillis;
> this.current = intervalMillis;
> this.maxIntervalMillis = maxIntervalMillis;
> }{code}
> The constructor of ProportionalRetryInterval requires maxIntervalMillis> 0.
> But TestHDFSContractMultipartUploader does not override the
> timeToBecomeConsistentMillis method, so maxIntervalMillis = 0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17233) Fix unit tes of HDFS-13934
fanrui created HADOOP-17233:
---
Summary: Fix unit tes of HDFS-13934
Key: HADOOP-17233
URL: https://issues.apache.org/jira/browse/HADOOP-17233
Project: Hadoop Common
Issue Type: Bug
Components: common
Reporter: fanrui
Assignee: fanrui
unit test failed of
org.apache.hadoop.fs.contract.AbstractContractMultipartUploaderTest#testConcurrentUploads.
Exception:
{code:java}
java.lang.IllegalArgumentExceptionjava.lang.IllegalArgumentException at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:127) at
org.apache.hadoop.test.LambdaTestUtils$ProportionalRetryInterval.(LambdaTestUtils.java:907)
at
org.apache.hadoop.fs.contract.AbstractContractMultipartUploaderTest.testConcurrentUploads(AbstractContractMultipartUploaderTest.java:815){code}
h3. Reason:
{code:java}
public ProportionalRetryInterval(int intervalMillis,
int maxIntervalMillis) {
Preconditions.checkArgument(intervalMillis > 0);
Preconditions.checkArgument(maxIntervalMillis > 0);
this.intervalMillis = intervalMillis;
this.current = intervalMillis;
this.maxIntervalMillis = maxIntervalMillis;
}{code}
The constructor of ProportionalRetryInterval requires maxIntervalMillis> 0. But
TestHDFSContractMultipartUploader does not override the
timeToBecomeConsistentMillis method, so maxIntervalMillis = 0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17232) Erasure Coding: Typo in document
Fei Hui created HADOOP-17232: Summary: Erasure Coding: Typo in document Key: HADOOP-17232 URL: https://issues.apache.org/jira/browse/HADOOP-17232 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 3.3.0 Environment: When review ec document and code, find the typo. Change "a erasure code" to "an erasure code" Reporter: Fei Hui Assignee: Fei Hui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [DISCUSS] GitHub PR link auto-posting to JIRA?
Thank you very much Ayush! In HDFS-15025, the ASF GitHub Bot is able to edit the "Time Tracking" fields on the right side. This indicates the permission should work now. I do not have a new PR filed. We can check incoming PRs and see if they get posted to JIRA automatically. On Wed, Aug 26, 2020 at 9:40 PM Ayush Saxena wrote: > Hi Mingliang, > I think this issue has been there for a couple of months, It used to work > earlier IIRC. > I tried checking a bit, I think ASF-GITHUB-BOT didn't have permissions, As > of now I added it as HDFS-Contributor-1(temporarily) and I just saw one > notification on HDFS-15025 from Github. > Can you check if that solves the issue? > > -Ayush > > On Thu, 27 Aug 2020 at 03:41, Mingliang Liu wrote: > >> Hi, >> >> I found that GitHub PR will not show up as "links" of the JIRA even if the >> PR subject starts with a JIRA number. >> >> Is this a known issue? I see this works for HBase projects, but not >> Hadoop. >> >> Thanks, >> >
Re: [DISCUSS] fate of branch-2.9
+1 for putting 2.9 lines to EOL. Let's focus on 2.10 releases for Hadoop 2. Also is there any plan for 2.10.1? It has been 11 months since 2.10 first release. Thanks, On Wed, Aug 26, 2020 at 10:57 PM Wei-Chiu Chuang wrote: > Bump up this thread after 6 months. > > Is anyone still interested in the 2.9 release line? Or are we good to start > the EOL process? The 2.9.2 was released in Nov 2018. > > I'd really like to see the community to converge to fewer release lines and > make more frequent releases in each line. > > Thanks, > Weichiu > > > On Fri, Mar 6, 2020 at 5:47 PM Wei-Chiu Chuang wrote: > > > I think that's a great suggestion. > > Currently, we make 1 minor release per year, and within each minor > release > > we bring up 1 thousand to 2 thousand commits in it compared with the > > previous one. > > I can totally understand it is a big bite for users to swallow. Having a > > more frequent release cycle, plus LTS and non-LTS releases should help > with > > this. (Of course we will need to make the release preparation much > easier, > > which is currently a pain) > > > > I am happy to discuss the release model further in the dev ML. LTS v.s. > > non-LTS is one suggestion. > > > > Another similar issue: In the past Hadoop strived to > > maintain compatibility. However, this is no longer sustainable as more > CVEs > > coming from our dependencies: netty, jetty, jackson ... etc. > > In many cases, updating the dependencies brings breaking changes. More > > recently, especially in Hadoop 3.x, I started to make the effort to > update > > dependencies much more frequently. How do users feel about this change? > > > > On Thu, Mar 5, 2020 at 7:58 AM Igor Dvorzhak > > wrote: > > > >> Maybe Hadoop will benefit from adopting a similar release and support > >> strategy as Java? I.e. designate some releases as LTS and support them > for > >> 2 (?) years (it seems that 2.7.x branch was de-facto LTS), other non-LTS > >> releases will be supported for 6 months (or until next release). This > >> should allow to reduce maintenance cost of non-LTS release and provide > >> conservative users desired stability by allowing them to wait for new > LTS > >> release and upgrading to it. > >> > >> On Thu, Mar 5, 2020 at 1:26 AM Rupert Mazzucco < > rupert.mazzu...@gmail.com> > >> wrote: > >> > >>> After recently jumping from 2.7.7 to 2.10 without issue myself, I vote > >>> for keeping only the 2.10 line. > >>> It would seem all other 2.x branches can upgrade to a 2.10.x easily if > >>> they feel like upgrading at all, > >>> unlike a jump to 3.x, which may require more planning. > >>> > >>> I also vote for having only one main 3.x branch. Why are there 3.1.x > and > >>> 3.2.x seemingly competing, > >>> and now 3.3.x? For a community that does not have the resources to > >>> manage multiple release lines, > >>> you guys sure like to multiply release lines a lot. > >>> > >>> Cheers > >>> Rupert > >>> > >>> Am Mi., 4. März 2020 um 19:40 Uhr schrieb Wei-Chiu Chuang > >>> : > >>> > Forwarding the discussion thread from the dev mailing lists to the > user > mailing lists. > > I'd like to get an idea of how many users are still on Hadoop 2.9. > Please share your thoughts. > > On Mon, Mar 2, 2020 at 6:30 PM Sree Vaddi > wrote: > > > +1 > > > > Sent from Yahoo Mail on Android > > > > On Mon, Mar 2, 2020 at 5:12 PM, Wei-Chiu Chuang > > > wrote: Hi, > > > > Following the discussion to end branch-2.8, I want to start a > > discussion > > around what's next with branch-2.9. I am hesitant to use the word > "end > > of > > life" but consider these facts: > > > > * 2.9.0 was released Dec 17, 2017. > > * 2.9.2, the last 2.9.x release, went out Nov 19 2018, which is more > > than > > 15 months ago. > > * no one seems to be interested in being the release manager for > 2.9.3. > > * Most if not all of the active Hadoop contributors are using Hadoop > > 2.10 > > or Hadoop 3.x. > > * We as a community do not have the cycle to manage multiple release > > line, > > especially since Hadoop 3.3.0 is coming out soon. > > > > It is perhaps the time to gradually reduce our footprint in Hadoop > > 2.x, and > > encourage people to upgrade to Hadoop 3.x > > > > Thoughts? > > > > > -- L
