Re: Next releases
Hi Arun, Another feature that would be relevant and got deferred was the symlink work (HADOOP-10020) that Colin and Andrew were working on. Can we include this in hadoop-2.3.0 also? thanks hari On Sun, Nov 10, 2013 at 2:07 PM, Alejandro Abdelnur wrote: > Arun, thanks for jumping on this. > > On hadoop branch-2.2. I've quickly scanned the commit logs starting from > the 2.2.0 release and I've found around 20 JIRAs that I like seeing in > 2.2.1. Not all of them are bugs but the don't shake anything and improve > usability. > > I presume others will have their own laundry lists as well and I wonder the > union of all of them how much adds up to the current 81 commits. > > How about splitting the JIRAs among a few contributors to assert there is > nothing risky in there? And if so get discuss getting rid of those commits > for 2.2.1. IMO doing that would be cheaper than selectively applying > commits on a fresh branch. > > Said this, I think we should get 2.2.1 out of the door before switching > main efforts to 2.3.0. I volunteer myself to drive 2.2.1 a release if ASAP > if you don't have the bandwidth at the moment for it. > > Cheers. > > Alejandro > > > > Commits in branch-2.2 that I'd like them to be in the 2.2.1 release: > > The ones prefixed with '*' technically are not bugs. > > YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed > containers. (Alejandro Abdelnur via Sandy Ryza) > YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza) > YARN-1044. used/min/max resources do not display info in the scheduler > page (Sangjin Lee via Sandy Ryza) > YARN-305. Fair scheduler logs too many "Node offered to app" messages. > (Lohit Vijayarenu via Sandy Ryza) > *MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy > Ryza) > YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps > switched. (Robert Kanter via Sandy Ryza) > YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text > file busy errors. (Sandy Ryza) > *MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write > out text files without separators (Sandy Ryza) > *YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza) > *YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza) > YARN-1330. Fair Scheduler: defaultQueueSchedulingPolicy does not take > effect (Sandy Ryza) > HDFS-5403. WebHdfs client cannot communicate with older WebHdfs servers > post HDFS-5306. Contributed by Aaron T. Myers. > *YARN-1335. Move duplicate code from FSSchedulerApp and FiCaSchedulerApp > into SchedulerApplication (Sandy Ryza) > *YARN-1333. Support blacklisting in the Fair Scheduler (Tsuyoshi Ozawa via > Sandy Ryza) > *MAPREDUCE-4680. Job history cleaner should only check timestamps of files > in old enough directories (Robert Kanter via Sandy Ryza) > YARN-1109. Demote NodeManager "Sending out status for container" logs to > debug (haosdent via Sandy Ryza) > *YARN-1321. Changed NMTokenCache to support both singleton and an instance > usage. Contributed by Alejandro Abdelnur > YARN-1343. NodeManagers additions/restarts are not reported as node > updates in AllocateResponse responses to AMs. (tucu) > YARN-1381. Same relaxLocality appears twice in exception message of > AMRMClientImpl#checkLocalityRelaxationConflict() (Ted Yu via Sandy Ryza) > HADOOP-9898. Set SO_KEEPALIVE on all our sockets. Contributed by Todd > Lipcon. > YARN-1388. Fair Scheduler page always displays blank fair share (Liyin > Liang via Sandy Ryza) > > > > On Fri, Nov 8, 2013 at 10:35 PM, Chris Nauroth >wrote: > > > Arun, what are your thoughts on test-only patches? I know I've been > > merging a lot of Windows test stabilization patches down to branch-2.2. > > These can't rightly be called blockers, but they do improve dev > > experience, and there is no risk to product code. > > > > Chris Nauroth > > Hortonworks > > http://hortonworks.com/ > > > > > > > > On Fri, Nov 8, 2013 at 1:30 AM, Steve Loughran > >wrote: > > > > > On 8 November 2013 02:42, Arun C Murthy wrote: > > > > > > > Gang, > > > > > > > > Thinking through the next couple of releases here, appreciate f/b. > > > > > > > > # hadoop-2.2.1 > > > > > > > > I was looking through commit logs and there is a *lot* of content > here > > > > (81 commits as on 11/7). Some are features/improvements and some are > > > fixes > > > > - it's really hard to distinguish what is important and what isn't. > > > > > > > > I propose we start with a blank slate (i.e. blow away branch-2.2 and > > > > start fresh from a copy of branch-2.2.0) and then be very careful > and > > > > meticulous about including only *blocker* fixes in branch-2.2. So, > most > > > of > > > > the content here comes via the next minor release (i.e. hadoop-2.3) > > > > > > > > In future, we continue to be *very* parsimonious about what gets > into
[jira] [Created] (HDFS-4873) callGetBlockLocations returns incorrect number of blocks for snapshotted files
Hari Mankude created HDFS-4873: -- Summary: callGetBlockLocations returns incorrect number of blocks for snapshotted files Key: HDFS-4873 URL: https://issues.apache.org/jira/browse/HDFS-4873 Project: Hadoop HDFS Issue Type: Bug Components: snapshots Affects Versions: 3.0.0 Reporter: Hari Mankude Assignee: Jing Zhao callGetBlockLocations() returns all the blocks of a file even when they are not present in the snap version -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3205) testHANameNodesWithFederation is failing in trunk
[ https://issues.apache.org/jira/browse/HDFS-3205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude resolved HDFS-3205. Resolution: Duplicate This is a dup of hdfs-2960 > testHANameNodesWithFederation is failing in trunk > - > > Key: HDFS-3205 > URL: https://issues.apache.org/jira/browse/HDFS-3205 > Project: Hadoop HDFS > Issue Type: Bug > Components: ha, name-node > Reporter: Hari Mankude >Assignee: Hari Mankude >Priority: Minor > > The test is failing with the error > org.junit.ComparisonFailure: expected: but > was: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3299) handle creation time also in journal service
handle creation time also in journal service Key: HDFS-3299 URL: https://issues.apache.org/jira/browse/HDFS-3299 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3293) Implement equals for storageinfo and journainfo class.
Implement equals for storageinfo and journainfo class. --- Key: HDFS-3293 URL: https://issues.apache.org/jira/browse/HDFS-3293 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Hari Mankude Priority: Minor Implement equals for storageinfo and journalinfo class. Also journalinfo class needs a toString() method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3274) Create a new journal_edits_dir key to support journal nodes
Create a new journal_edits_dir key to support journal nodes Key: HDFS-3274 URL: https://issues.apache.org/jira/browse/HDFS-3274 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude Assignee: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3257) Fix synchronization issues with journal service
[ https://issues.apache.org/jira/browse/HDFS-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude resolved HDFS-3257. Resolution: Invalid closing the jira since there is only one thread as per Suresh's comment. > Fix synchronization issues with journal service > --- > > Key: HDFS-3257 > URL: https://issues.apache.org/jira/browse/HDFS-3257 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node > Reporter: Hari Mankude >Assignee: Hari Mankude > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3257) Fix synchronization issues with journal service
Fix synchronization issues with journal service --- Key: HDFS-3257 URL: https://issues.apache.org/jira/browse/HDFS-3257 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude Assignee: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3217) ZKFC should restart NN when healthmonitor gets a SERVICE_NOT_RESPONDING exception
ZKFC should restart NN when healthmonitor gets a SERVICE_NOT_RESPONDING exception - Key: HDFS-3217 URL: https://issues.apache.org/jira/browse/HDFS-3217 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3213) JournalDaemon (server) should persist the cluster id and nsid in the storage directory
JournalDaemon (server) should persist the cluster id and nsid in the storage directory -- Key: HDFS-3213 URL: https://issues.apache.org/jira/browse/HDFS-3213 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3205) testHANameNodesWithFederation is failing in trunk
testHANameNodesWithFederation is failing in trunk - Key: HDFS-3205 URL: https://issues.apache.org/jira/browse/HDFS-3205 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor The test is failing with the error org.junit.ComparisonFailure: expected: but was: -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3192) Active NN should exit when it has not received a getServiceStatus() rpc from ZKFC for timeout secs
Active NN should exit when it has not received a getServiceStatus() rpc from ZKFC for timeout secs -- Key: HDFS-3192 URL: https://issues.apache.org/jira/browse/HDFS-3192 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3191) Active NN should exit when it cannot write to quorum number of Journal Daemons
Active NN should exit when it cannot write to quorum number of Journal Daemons -- Key: HDFS-3191 URL: https://issues.apache.org/jira/browse/HDFS-3191 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3186) Sync lagging journal service from the active journal service
Sync lagging journal service from the active journal service Key: HDFS-3186 URL: https://issues.apache.org/jira/browse/HDFS-3186 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3185) Setup configuration for Journal Manager and Journal Services
Setup configuration for Journal Manager and Journal Services Key: HDFS-3185 URL: https://issues.apache.org/jira/browse/HDFS-3185 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude Assignee: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3126) Journal stream from the namenode to backup needs to have a timeout
Journal stream from the namenode to backup needs to have a timeout -- Key: HDFS-3126 URL: https://issues.apache.org/jira/browse/HDFS-3126 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Hari Mankude Assignee: Hari Mankude -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-3009) DFSclient islocaladdress() can use similar routine in netutils
DFSclient islocaladdress() can use similar routine in netutils -- Key: HDFS-3009 URL: https://issues.apache.org/jira/browse/HDFS-3009 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Trivial isLocalAddress() in dfsclient can use similar function in netutils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2955) IllegalStateException during standby startup in getCurSegmentTxId
IllegalStateException during standby startup in getCurSegmentTxId - Key: HDFS-2955 URL: https://issues.apache.org/jira/browse/HDFS-2955 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Assignee: Hari Mankude During standby restarts, a new routine getTransactionsSinceLastLogRoll() has been introduced for metrics which is calling getCurSegmentTxId(). checkstate() in getCurSegmentTxId() assumes that log is opened for writing and this is not the case in standby. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2865) Standby namenode gets a "cannot lock storage" exception during startup
[ https://issues.apache.org/jira/browse/HDFS-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude resolved HDFS-2865. Resolution: Invalid > Standby namenode gets a "cannot lock storage" exception during startup > -- > > Key: HDFS-2865 > URL: https://issues.apache.org/jira/browse/HDFS-2865 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Hari Mankude >Assignee: Hari Mankude > > Standby NN is restarted. This is a follow-on to hdfs-2863. In this setup, > dfs.edits.dir is different from dfs.shared.edits.dir. During startup, standby > NN fails to acquire lock on the dfs.edits.dir. If standby NN is restarted > again, it seems to work fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2866) Standby does not start up due to a gap in transaction id
[ https://issues.apache.org/jira/browse/HDFS-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude resolved HDFS-2866. Resolution: Duplicate hdfs-2874 > Standby does not start up due to a gap in transaction id > > > Key: HDFS-2866 > URL: https://issues.apache.org/jira/browse/HDFS-2866 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Hari Mankude > Assignee: Hari Mankude >Priority: Critical > > Standby notices a gap in the transaction id in the shared.edits directory. > The transactions in dfs.edits.dir does not seem to have the gap. The gap > happens during a failover. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2914) HA Standby stuck in safemode when shared edits directory is bounced
HA Standby stuck in safemode when shared edits directory is bounced --- Key: HDFS-2914 URL: https://issues.apache.org/jira/browse/HDFS-2914 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude When shared edits dir is bounced, standby NN is put into safemode by the NameNodeResourceMonitor(). However, there is no path for it to exit out of safe mode when shared edits dir reappears. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2903) NPE if shared edits directory is not available during failover
[ https://issues.apache.org/jira/browse/HDFS-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude resolved HDFS-2903. Resolution: Duplicate Dup of HDFS-2905 > NPE if shared edits directory is not available during failover > -- > > Key: HDFS-2903 > URL: https://issues.apache.org/jira/browse/HDFS-2903 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Hari Mankude > Assignee: Hari Mankude > > Journalset has a null pointer exception if shared edits directory is not > available during HA state transition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2903) NPE if shared edits directory is not available during failover
NPE if shared edits directory is not available during failover -- Key: HDFS-2903 URL: https://issues.apache.org/jira/browse/HDFS-2903 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Assignee: Hari Mankude Journalset has a null pointer exception if shared edits directory is not available during HA state transition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2866) Standby does not start up due to a gap in transaction id
Standby does not start up due to a gap in transaction id Key: HDFS-2866 URL: https://issues.apache.org/jira/browse/HDFS-2866 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Priority: Critical Standby notices a gap in the transaction id in the shared.edits directory. The transactions in dfs.edits.dir does not seem to have the gap. The gap happens during a failover. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2865) Standby namenode gets a "cannot lock storage" exception during startup
Standby namenode gets a "cannot lock storage" exception during startup -- Key: HDFS-2865 URL: https://issues.apache.org/jira/browse/HDFS-2865 Project: Hadoop HDFS Issue Type: Bug Reporter: Hari Mankude Assignee: Hari Mankude Standby NN is restarted. This is a follow-on to hdfs-2863. In this setup, dfs.edits.dir is different from dfs.shared.edits.dir. During startup, standby NN fails to acquire lock on the dfs.edits.dir. If standby NN is restarted again, it seems to work fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude reopened HDFS-2802: I (We) are well aware of hdfs-233. This jira was opened to provide comprehensive snapshot solution (both RW/RO support) for HDFS. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: name-node > Reporter: Hari Mankude >Assignee: Hari Mankude > Fix For: 0.24.0 > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2802) Support for RW/RO snapshots in HDFS
Support for RW/RO snapshots in HDFS --- Key: HDFS-2802 URL: https://issues.apache.org/jira/browse/HDFS-2802 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Reporter: Hari Mankude Assignee: Hari Mankude Fix For: 0.24.0 Snapshots are point in time images of parts of the filesystem or the entire filesystem. Snapshots can be a read-only or a read-write point in time copy of the filesystem. There are several use cases for snapshots in HDFS. I will post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2783) HDFS jira for changes related to HADOOP-7970
HDFS jira for changes related to HADOOP-7970 Key: HDFS-2783 URL: https://issues.apache.org/jira/browse/HDFS-2783 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Create a separate hdfs patch for HDFS changes related to Hadoop-7970 to avoid test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2780) All HAServiceProtocol methods have to throw IOException since they are invoked over the network
All HAServiceProtocol methods have to throw IOException since they are invoked over the network --- Key: HDFS-2780 URL: https://issues.apache.org/jira/browse/HDFS-2780 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Assignee: Hari Mankude HAServiceProtocol methods have to throw ioexception in addition to other exceptions since they are RPCs which are invoked over the network. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2756) Warm standby does not read the in_progress edit log
Warm standby does not read the in_progress edit log Key: HDFS-2756 URL: https://issues.apache.org/jira/browse/HDFS-2756 Project: Hadoop HDFS Issue Type: Bug Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Hari Mankude Warm standby does not read the in_progress edit log. This could result in standby taking a long time to become the primary during a failover scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2753) Standby namenode stuck in safenode during a failover
Standby namenode stuck in safenode during a failover
Key: HDFS-2753
URL: https://issues.apache.org/jira/browse/HDFS-2753
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Hari Mankude
Assignee: Hari Mankude
teragen is run to generate write traffic. A manual failover is initiated by
killing namenode process. namenode which was killed is then restarted and it
comes up as a standby. However, the standby never exits out of the safemode.
Looking at the standby logs, it looks like the standby namenode gets
addStoredBlock immediately after data node registration which results in
numBlocks getting incremented. There is an optimization in processReport()
which causes the follow on complete block report to be ignored when NN is in
safemode. So, NN never exits out of the safemode.
2012-01-05 18:57:46,030 INFO hdfs.StateChange
(DatanodeManager.java:registerDatanode(573)) - BLOCK*
NameSystem.registerDatanode: node registration from 98.137.233.235:50010
storage DS-526656430-98.137.233.235-50010-1325723536492
2012-01-05 18:57:46,033 INFO net.NetworkTopology
(NetworkTopology.java:add(344)) - Adding a new node:
/default-rack/98.137.233.235:50010
2012-01-05 18:57:46,033 INFO namenode.FSNamesystem
(FSNamesystem.java:checkMode(3411)) - DID NOT call initialize at 1
2012-01-05 18:57:46,034 INFO hdfs.StateChange
(DatanodeManager.java:registerDatanode(573)) - BLOCK*
NameSystem.registerDatanode: node registration from 98.137.233.237:50010
storage DS-1961520590-98.137.233.237-50010-1325725253057
2012-01-05 18:57:46,034 INFO net.NetworkTopology
(NetworkTopology.java:add(344)) - Adding a new node:
/default-rack/98.137.233.237:50010
2012-01-05 18:57:46,042 INFO namenode.FSNamesystem
(FSNamesystem.java:checkMode(3411)) - DID NOT call initialize at 1
2012-01-05 18:57:46,045 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(1775)) - BLOCK* addStoredBlock: blockMap
updated: 98.137.233.235:50010 is added to
blk_-3183325095022454724_1172{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[98.137.233.235:50010|FINALIZED]]} size 0
2012-01-05 18:57:46,046 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(1775)) - BLOCK* addStoredBlock: blockMap
updated: 98.137.233.235:50010 is added to
blk_5617057825952660916_1173{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[98.137.233.235:50010|FINALIZED]]} size 0
2012-01-05 18:57:46,046 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(1775)) - BLOCK* addStoredBlock: blockMap
updated: 98.137.233.237:50010 is added to
blk_-3183325095022454724_1172{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[98.137.233.235:50010|FINALIZED],
ReplicaUnderConstruction[98.137.233.237:50010|FINALIZED]]} size 0
2012-01-05 18:57:46,046 INFO hdfs.StateChange
(BlockManager.java:addStoredBlock(1775)) - BLOCK* addStoredBlock: blockMap
updated: 98.137.233.237:50010 is added to
blk_5617057825952660916_1173{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[98.137.233.235:50010|FINALIZED],
ReplicaUnderConstruction[98.137.233.237:50010|FINALIZED]]} size 0
2012-01-05 18:57:46,049 INFO hdfs.StateChange
(BlockManager.java:processReport(1365)) - BLOCK* processReport: discarded
non-initial block report from 98.137.233.235:50010 because namenode still in
startup phase
2012-01-05 18:57:46,049 INFO hdfs.StateChange
(BlockManager.java:processReport(1365)) - BLOCK* processReport: discarded
non-initial block report from 98.137.233.237:50010 because namenode still in
startup phase
2012-01-05 18:58:05,167 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1175,
Queuing blockReceivedAndDeleted message
2012-01-05 18:58:05,168 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1175,
Queuing blockReceivedAndDeleted message
2012-01-05 18:58:06,634 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1176,
Queuing blockReceivedAndDeleted message
2012-01-05 18:58:06,636 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1176,
Queuing blockReceivedAndDeleted message
2012-01-05 18:58:08,097 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1177,
Queuing blockReceivedAndDeleted message
2012-01-05 18:58:08,097 INFO namenode.NameNode
(NameNodeRpcServer.java:blockReceivedAndDeleted(894)) - Required GS=1177,
Queuing blockReceivedAndDeleted message
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira
[jira] [Created] (HDFS-2650) Replace @inheritDoc with @Override
Replace @inheritDoc with @Override --- Key: HDFS-2650 URL: https://issues.apache.org/jira/browse/HDFS-2650 Project: Hadoop HDFS Issue Type: Improvement Reporter: Hari Mankude Assignee: Hari Mankude Priority: Minor Fix For: 0.24.0 @Override provides both javadoc from superclass and compile time detection of overridden method deletion from superclass. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2564) Cleanup unnecessary exceptions thrown and unnecessary casts
Cleanup unnecessary exceptions thrown and unnecessary casts --- Key: HDFS-2564 URL: https://issues.apache.org/jira/browse/HDFS-2564 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, hdfs client, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Priority: Minor Fix For: 0.24.0 Attachments: HDFS-2564.txt Cleaning up some of the java files with unnecessary exceptions and casts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
