NN fencing script for kubernetes managed clusters

Hi All, We recently encountered a scenario where clients were reading from previously active NN. The active namenode was running at almost 100% memory that the namenode was alive but in a constant GC loop. The zookeeper session expired and we have an automatic failover setup which automatically sel

[jira] [Resolved] (HDFS-17299) HDFS is not rack failure tolerant while creating a new file.

[ https://issues.apache.org/jira/browse/HDFS-17299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh Shah resolved HDFS-17299. - Fix Version/s: 3.4.1 3.5.0 Resolution: Fixed > HDFS is not r

[jira] [Created] (HDFS-17409) TestDFSAdmin has couple of flaky tests.

Rushabh Shah created HDFS-17409: --- Summary: TestDFSAdmin has couple of flaky tests. Key: HDFS-17409 URL: https://issues.apache.org/jira/browse/HDFS-17409 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-17299) HDFS is not rack failure tolerant while creating a new file.

Rushabh Shah created HDFS-17299: --- Summary: HDFS is not rack failure tolerant while creating a new file. Key: HDFS-17299 URL: https://issues.apache.org/jira/browse/HDFS-17299 Project: Hadoop HDFS

[jira] [Created] (HDFS-17188) Data loss in our production clusters due to missing HDFS-16540

Rushabh Shah created HDFS-17188: --- Summary: Data loss in our production clusters due to missing HDFS-16540 Key: HDFS-17188 URL: https://issues.apache.org/jira/browse/HDFS-17188 Project: Hadoop HDFS

[jira] [Created] (HDFS-16829) Delay deleting blocks with older generation stamp until the block is fully replicated.

Rushabh Shah created HDFS-16829: --- Summary: Delay deleting blocks with older generation stamp until the block is fully replicated. Key: HDFS-16829 URL: https://issues.apache.org/jira/browse/HDFS-16829

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

roup*: jira-users > > > Thanks. > > --Yongjun > > On Fri, Jun 1, 2018 at 8:48 AM, Rushabh Shah wrote: > >> Hi Yongjun, >> Thanks for all the effort you put in for this release. >> >>> https://issues.apache.org/jira/issues/?filter=12343997 >> >

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

Hi Yongjun, Thanks for all the effort you put in for this release. > https://issues.apache.org/jira/issues/?filter=12343997 Even after logging in, I am not able to see any jiras. It says "The requested filter doesn't exist or is private." On Fri, Jun 1, 2018 at 10:41 AM, Yongjun Zhang wrote:

Re: [VOTE] Release Apache Hadoop 2.9.1 (RC0)

Hi Chen, I am so sorry to bring this up now but there are 16 tests failing in hadoop-distcp project. I have opened a ticket and cc'ed Junping since he is branch-2.8 committer but I missed to ping you. IMHO we should fix the unit tests before we release but I would leave upto other members to give

Re: [VOTE] Merge HDDS (HDFS-7240) *code* into trunk

Hi All, Thanks Jitendra for initiating the merge vote. Just wanted to understand the policy for merge votes. According to this page: http://hadoop.apache.org/bylaws.html, I assume merge votes falls under "Adoption of new codebase" and only PMC members vote are considered binding. Is my understandin

Hdfs build on branch-2 are failing.

ime Far from being an expert on ssl, do we need to change the truststore on slave also ? Appreciate if anyone can help fixing this. Thanks, Rushabh Shah.

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

+1 (non-binding). Built from source and deployed it on pseudo cluster. Ran sample fs shell commands. Ran coupe of jobs: sleep and pi. Tested on java: 1.8.0_131 version. Thanks Andrew for all your efforts in getting us here. :) On Tue, Nov 14, 2017 at 3:34 PM, Andrew Wang wrote: > Hi folks, >

HDFS Pre-commit taking more than 24 hours.

03.gq1.ygridcore.net Is anyone else facing this issue ? Thanks, Rushabh Shah

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

Thank you Andrew for all your efforts. +1 non-binding. Built from source. Started 1 node pseudo cluster. Ran sleep and pi job. Except for YARN-6999 I didn't face any issues. On Thu, Sep 28, 2017 at 7:04 PM, Andrew Wang wrote: > Hi all, > > Let m

HDFS pre-commit builds are unstable on branch-2

Hi All, Recently hdfs precommit build for branch-2 is unstable, marking many (322 to be precise) unit tests as failed or timed out. I have created tracking ticket https://issues.apache.org/jira/browse/HDFS-12548 with additional details. Appreciate your inputs on that ticket.

Re: [DISCUSS] Branches and versions for Hadoop 3

+1 from me too for the branching proposal. On Fri, Aug 25, 2017 at 12:56 PM, Eric Payne < eric.payne1...@yahoo.com.invalid> wrote: > +1 for this branching proposal.-Eric > > > From: Andrew Wang > To: "common-...@hadoop.apache.org" ; " > mapreduce-...@hadoop.apache.org" ; " > hdfs-dev@hado

Re: [VOTE] Release Apache Hadoop 2.7.3 RC2

Thanks Vinod for another release for 2.7.3.+1 (non-binding) Downloaded source tarball. Compiled and built from source.Deployed a pseudo distributed cluster.Ran some sample jobs (sleep, pi). All ran successfully. On Wednesday, August 17, 2016 9:05 PM, Vinod Kumar Vavilapalli wrote: Hi

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

Thanks Vinod for all the release work ! +1 (non-binding). * Downloaded from source and built it.* Deployed a pseudo distributed cluster. * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works fine. On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli wrote: H

Re: Adding functionality based on mover

ny jira to yourself.One of the committers will need to add you as a contributor to HDFS project.But all the hadoop developers will be able to see your jira. Hope this helps. Thanks,Rushabh Shah. On Sunday, June 12, 2016 5:23 PM, Justyna Kałużka wrote: Hi everyone, As my master thesis I'm

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

+1 (non-binding)- Downloaded source tarball.- Successfully built from source tarball.- Deployed on a single node cluster. -  Ran sample MR jobs and it ran successfully. Thanks a lot Vinod for your efforts ! On Tuesday, January 19, 2016 8:34 PM, Akira AJISAKA wrote: +1 (binding) - Dow

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

+1 non-bindingBuilt from source, stood up a single node cluster, run multiple jobs i.e sleep and pi job.Both ran fine. Thanks Vinod  for driving 2.7.0 deployment. On Wednesday, April 15, 2015 3:44 PM, Karthik Kambatla wrote: Thanks for putting this together, Vinod. Stood up a pse