[jira] [Created] (HADOOP-18127) Backport HADOOP-13055 into branch-2.10
Konstantin Shvachko created HADOOP-18127: Summary: Backport HADOOP-13055 into branch-2.10 Key: HADOOP-18127 URL: https://issues.apache.org/jira/browse/HADOOP-18127 Project: Hadoop Common Issue Type: Sub-task Components: viewfs Reporter: Konstantin Shvachko HADOOP-13055 introduce linkMergeSlash and linkFallback for ViewFileSystem. Would be good to backport it to branch-2.10 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17999) No-op implementation of setWriteChecksum and setVerifyChecksum in ViewFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-17999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-17999. -- Fix Version/s: 3.4.0 2.10.2 3.2.4 3.3.3 Hadoop Flags: Reviewed Resolution: Fixed I just committed this to all supported branches. Thank you [~abhishekd]. > No-op implementation of setWriteChecksum and setVerifyChecksum in > ViewFileSystem > > > Key: HADOOP-17999 > URL: https://issues.apache.org/jira/browse/HADOOP-17999 > Project: Hadoop Common > Issue Type: Bug >Reporter: Abhishek Das >Assignee: Abhishek Das >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently setVerifyChecksum and setWriteChecksum initializes all target file > systems which causes delay in hadoop shell copy commands such as get, put, > copyFromLocal etc. > This also eventually causes OOM. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16532) Fix TestViewFsTrash to use the correct homeDir.
[
https://issues.apache.org/jira/browse/HADOOP-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko resolved HADOOP-16532.
--
Fix Version/s: 3.2.4
3.3.2
2.10.2
3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
I just committed this. Thank you [~xinglin].
> Fix TestViewFsTrash to use the correct homeDir.
> ---
>
> Key: HADOOP-16532
> URL: https://issues.apache.org/jira/browse/HADOOP-16532
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Xing Lin
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 2.10.2, 3.3.2, 3.2.4
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> the test {{TestViewFsTrash}} uses the .Trash directory under the current
> user's home dir so
> * fails in test setups which block writing to it (jenkins)
> * fails when users have real trash in there
> * may fail if there are parallel test runs.
> the home dir should be under some test path of the build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17819) Add extensions to ProtobufRpcEngine RequestHeaderProto
[ https://issues.apache.org/jira/browse/HADOOP-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-17819. -- Fix Version/s: 3.2.3 2.10.2 3.4.0 Hadoop Flags: Reviewed Assignee: Hector Sandoval Chaverri Resolution: Fixed I just committed this. Thank you [~hchaverri] > Add extensions to ProtobufRpcEngine RequestHeaderProto > -- > > Key: HADOOP-17819 > URL: https://issues.apache.org/jira/browse/HADOOP-17819 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Hector Sandoval Chaverri >Assignee: Hector Sandoval Chaverri >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.3 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The header used in ProtobufRpcEngine messages doesn't allow for new > properties to be added by child classes. We can add a range of extensions > that can be useful for proto classes that need to extend RequestHeaderProto. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17028) ViewFS should initialize target filesystems lazily
[ https://issues.apache.org/jira/browse/HADOOP-17028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-17028. -- Fix Version/s: 3.3.2 3.2.3 2.10.2 3.4.0 3.1.5 Hadoop Flags: Reviewed Resolution: Fixed > ViewFS should initialize target filesystems lazily > -- > > Key: HADOOP-17028 > URL: https://issues.apache.org/jira/browse/HADOOP-17028 > Project: Hadoop Common > Issue Type: Bug > Components: client-mounts, fs, viewfs >Affects Versions: 3.2.1 >Reporter: Uma Maheswara Rao G >Assignee: Abhishek Das >Priority: Major > Labels: pull-request-available > Fix For: 3.1.5, 3.4.0, 2.10.2, 3.2.3, 3.3.2 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Currently viewFS initialize all configured target filesystems when > viewfs#init itself. > Some target file system initialization involve creating heavy objects and > proxy connections. Ex: DistributedFileSystem#initialize will create DFSClient > object which will create proxy connections to NN etc. > For example: if ViewFS configured with 10 target fs with hdfs uri and 2 > targets with s3a. > If one of the client only work with s3a target, But ViewFS will initialize > all targets irrespective of what clients interested to work with. That means, > here client will create 10 DFS initializations and 2 s3a initializations. Its > unnecessary to have DFS initialization here. So, it will be a good idea to > initialize the target fs only when first time usage call come to particular > target fs scheme. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17501) Fix logging typo in ShutdownHookManager
Konstantin Shvachko created HADOOP-17501:
Summary: Fix logging typo in ShutdownHookManager
Key: HADOOP-17501
URL: https://issues.apache.org/jira/browse/HADOOP-17501
Project: Hadoop Common
Issue Type: Improvement
Components: common
Reporter: Konstantin Shvachko
Three log messages in {{ShutdownHookManager}} have a typo, saying
"ShutdownHookManger". SHould be "ShutdownHookManager"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17477) [SBN read] Implement msync() for ViewFS
Konstantin Shvachko created HADOOP-17477:
Summary: [SBN read] Implement msync() for ViewFS
Key: HADOOP-17477
URL: https://issues.apache.org/jira/browse/HADOOP-17477
Project: Hadoop Common
Issue Type: Improvement
Components: viewfs
Reporter: Konstantin Shvachko
We should implement {{FileSystem.mscyn()}} for {{ViewFS}} and
{{ViewFileSystem}}.
It should call msync() on all volumes for which observer reads are enabled and
{{dfs.namenode.state.context.enabled = true}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17026) TestRefreshCallQueue is failing due to changed CallQueue constructor
Konstantin Shvachko created HADOOP-17026: Summary: TestRefreshCallQueue is failing due to changed CallQueue constructor Key: HADOOP-17026 URL: https://issues.apache.org/jira/browse/HADOOP-17026 Project: Hadoop Common Issue Type: Bug Components: test Reporter: Konstantin Shvachko -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-15325) Make Configuration#getPasswordFromCredentialsProvider() a public API
[ https://issues.apache.org/jira/browse/HADOOP-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-15325. -- Resolution: Implemented This has been incorporated into HADOOP-12862. > Make Configuration#getPasswordFromCredentialsProvider() a public API > > > Key: HADOOP-15325 > URL: https://issues.apache.org/jira/browse/HADOOP-15325 > Project: Hadoop Common > Issue Type: Improvement > Components: conf >Affects Versions: 2.6.0 >Reporter: Wei-Chiu Chuang >Assignee: Zsolt Venczel >Priority: Major > > HADOOP-10607 added a public API Configuration.getPassword() which reads > passwords from credential provider and then falls back to reading from > configuration if one is not available. > This API has been used throughout Hadoop codebase and downstream > applications. It is understandable for old password configuration keys to > fallback to configuration to maintain backward compatibility. But for new > configuration passwords that don't have legacy, there should be an option to > _not_ fallback, because storing passwords in configuration is considered a > bad security practice. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-12751) While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple
[ https://issues.apache.org/jira/browse/HADOOP-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reopened HADOOP-12751: -- > While using kerberos Hadoop incorrectly assumes names with '@' to be > non-simple > --- > > Key: HADOOP-12751 > URL: https://issues.apache.org/jira/browse/HADOOP-12751 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.7.2 > Environment: kerberos >Reporter: Bolke de Bruin >Assignee: Bolke de Bruin >Priority: Critical > Labels: kerberos > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: 0001-HADOOP-12751-leave-user-validation-to-os.patch, > 0001-Remove-check-for-user-name-characters-and.patch, > 0002-HADOOP-12751-leave-user-validation-to-os.patch, > 0003-HADOOP-12751-leave-user-validation-to-os.patch, > 0004-HADOOP-12751-leave-user-validation-to-os.patch, > 0005-HADOOP-12751-leave-user-validation-to-os.patch, > 0006-HADOOP-12751-leave-user-validation-to-os.patch, > 0007-HADOOP-12751-leave-user-validation-to-os.patch, > 0007-HADOOP-12751-leave-user-validation-to-os.patch, > 0008-HADOOP-12751-leave-user-validation-to-os.patch, > 0008-HADOOP-12751-leave-user-validation-to-os.patch, HADOOP-12751-009.patch > > > In the scenario of a trust between two directories, eg. FreeIPA (ipa.local) > and Active Directory (ad.local) users can be made available on the OS level > by something like sssd. The trusted users will be of the form 'user@ad.local' > while other users are will not contain the domain. Executing 'id -Gn > user@ad.local' will successfully return the groups the user belongs to if > configured correctly. > However, it is assumed by Hadoop that users of the format with '@' cannot be > correct. This code is in KerberosName.java and seems to be a validator if the > 'auth_to_local' rules are applied correctly. > In my opinion this should be removed or changed to a different kind of check > or maybe logged as a warning while still proceeding, as the current behavior > limits integration possibilities with other standard tools. > Workaround are difficult to apply (by having a rewrite by system tools to for > example user_ad_local) due to down stream consequences. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14689) Dummy jira for testing barnch-2.7 build.
Konstantin Shvachko created HADOOP-14689: Summary: Dummy jira for testing barnch-2.7 build. Key: HADOOP-14689 URL: https://issues.apache.org/jira/browse/HADOOP-14689 Project: Hadoop Common Issue Type: Task Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Will add dummy patches to test Jenkins PreCommit-HADOOP-Build failures on branch-2.7. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14686) Branch-2.7 .gitignore is out of date
[ https://issues.apache.org/jira/browse/HADOOP-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-14686. -- Resolution: Fixed Hadoop Flags: Reviewed Also updated CHANGES.txt. I just committed this to branch 2.7. Thank you [~busbey]. > Branch-2.7 .gitignore is out of date > > > Key: HADOOP-14686 > URL: https://issues.apache.org/jira/browse/HADOOP-14686 > Project: Hadoop Common > Issue Type: Bug > Components: build, precommit >Affects Versions: 2.7.4 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.7.4 > > Attachments: HADOOP-14686-branch-2.7.v0.patch, > HADOOP-14686-branch-2.7.v1.patch > > > .gitignore is out of date on branch-2.7, which is causing issues in precommit > checks for that branch. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-14686) Branch-2.7 .gitignore is out of date
[
https://issues.apache.org/jira/browse/HADOOP-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko reopened HADOOP-14686:
--
Will have to revert this, because it also removed {{contract-test-options.xml}}
from .gitignore, which could be considered as incompatible change for 2.7.4.
> Branch-2.7 .gitignore is out of date
>
>
> Key: HADOOP-14686
> URL: https://issues.apache.org/jira/browse/HADOOP-14686
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, precommit
>Affects Versions: 2.7.4
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.7.4
>
> Attachments: HADOOP-14686-branch-2.7.v0.patch
>
>
> .gitignore is out of date on branch-2.7, which is causing issues in precommit
> checks for that branch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14676) Wrong default value for "fs.du.interval"
Konstantin Shvachko created HADOOP-14676:
Summary: Wrong default value for "fs.du.interval"
Key: HADOOP-14676
URL: https://issues.apache.org/jira/browse/HADOOP-14676
Project: Hadoop Common
Issue Type: Bug
Components: common, conf, fs
Affects Versions: 2.6.1
Reporter: Konstantin Shvachko
According to {{core-default.xml}} the default value of {{fs.du.interval = 60
sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The problem
is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}} instead of the
correct one {{CommonConfigurationKeysPublic.FS_DU_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14446) Backport HADOOP-7851 to branch 2.7
[ https://issues.apache.org/jira/browse/HADOOP-14446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-14446. -- Resolution: Fixed Hadoop Flags: Reviewed I just committed this to branch-2, 2.8, 2.7. Thank you [~elgoiri] for backport. > Backport HADOOP-7851 to branch 2.7 > -- > > Key: HADOOP-14446 > URL: https://issues.apache.org/jira/browse/HADOOP-14446 > Project: Hadoop Common > Issue Type: Bug >Reporter: Inigo Goiri >Assignee: Inigo Goiri > Attachments: HADOOP-7851-branch-2.7.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13902) Update documentation to reflect IPv6 Hadoop cluster setup
Konstantin Shvachko created HADOOP-13902: Summary: Update documentation to reflect IPv6 Hadoop cluster setup Key: HADOOP-13902 URL: https://issues.apache.org/jira/browse/HADOOP-13902 Project: Hadoop Common Issue Type: Sub-task Components: documentation Reporter: Konstantin Shvachko Document IPv6 cluster setup. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13290) Appropriate use of generics in FairCallQueue
Konstantin Shvachko created HADOOP-13290:
Summary: Appropriate use of generics in FairCallQueue
Key: HADOOP-13290
URL: https://issues.apache.org/jira/browse/HADOOP-13290
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 2.6.0
Reporter: Konstantin Shvachko
# {{BlockingQueue}} is intermittently used with and without generic parameters
in {{FairCallQueue}} class. Should be parameterized.
# Same for {{FairCallQueue}}. Should be parameterized. Could be a bit more
tricky for that one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13289) Remove unused variables in TestFairCallQueue
Konstantin Shvachko created HADOOP-13289:
Summary: Remove unused variables in TestFairCallQueue
Key: HADOOP-13289
URL: https://issues.apache.org/jira/browse/HADOOP-13289
Project: Hadoop Common
Issue Type: Bug
Components: test
Reporter: Konstantin Shvachko
# Remove unused member {{alwaysZeroScheduler}} and related initialization in
{{TestFairCallQueue}}
# Remove unused local vriable {{sched}} in
{{testOfferSucceedsWhenScheduledLowPriority()}}
And propagate to applicable release branches.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-10641) Introduce Coordination Engine
Konstantin Shvachko created HADOOP-10641: Summary: Introduce Coordination Engine Key: HADOOP-10641 URL: https://issues.apache.org/jira/browse/HADOOP-10641 Project: Hadoop Common Issue Type: New Feature Affects Versions: 3.0.0 Reporter: Konstantin Shvachko Coordination Engine (CE) is a system, which allows to agree on a sequence of events in a distributed system. In order to be reliable CE should be distributed by itself. Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and have different implementations, depending on use cases, reliability, availability, and performance requirements. CE should have a common API, so that it could serve as a pluggable component in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909). First implementation is proposed to be based on ZooKeeper. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HADOOP-8357) Restore security in Hadoop 0.22 branch
[ https://issues.apache.org/jira/browse/HADOOP-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-8357. - Resolution: Fixed Fix Version/s: 0.22.1 Yes. This was completed. Resolving. > Restore security in Hadoop 0.22 branch > -- > > Key: HADOOP-8357 > URL: https://issues.apache.org/jira/browse/HADOOP-8357 > Project: Hadoop Common > Issue Type: Task > Components: security >Affects Versions: 0.22.0 >Reporter: Konstantin Shvachko >Assignee: Benoy Antony > Fix For: 0.22.1 > > Attachments: performance_22_vs_22sec.pdf, > performance_22_vs_22sec_vs_22secon.pdf, SecurityTestPlan_results.pdf, > test_patch_results > > > This is to track changes for restoring security in 0.22 branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8558) Hadoop RPC does not allow protocol extension with common interfaces.
[ https://issues.apache.org/jira/browse/HADOOP-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-8558. - Resolution: Not A Problem Fix Version/s: 3.0.0 Assignee: Konstantin Shvachko Resolving as not a problem. > Hadoop RPC does not allow protocol extension with common interfaces. > > > Key: HADOOP-8558 > URL: https://issues.apache.org/jira/browse/HADOOP-8558 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.0.0-alpha >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Fix For: 3.0.0 > > Attachments: TestProtocolExtension.java, TestProtocolExtension.java > > > Hadoop RPC fails if MyProtocol extends an interface, which is not a > VersionedProtocol even if MyProtocol extends also VersionedProtocol. The > reason is that Invocation uses Method.getDeclaringClass(), which returns the > interface class rather than the class of MyProtocol. > This is incompatible with former versions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-8558) Hadoop RPC does not allow protocol extension with common interfaces.
Konstantin Shvachko created HADOOP-8558: --- Summary: Hadoop RPC does not allow protocol extension with common interfaces. Key: HADOOP-8558 URL: https://issues.apache.org/jira/browse/HADOOP-8558 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Konstantin Shvachko Hadoop RPC fails if MyProtocol extends an interface, which is not a VersionedProtocol even if MyProtocol extends also VersionedProtocol. The reason is that Invocation uses Method.getDeclaringClass(), which returns the interface class rather than the class of MyProtocol. This is incompatible with former versions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8383) TestKerberosAuthenticator fails
[ https://issues.apache.org/jira/browse/HADOOP-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-8383. - Resolution: Fixed Target Version/s: 0.22.1 Hadoop Flags: Reviewed +1 I just committed this to branch 0.22.1. Thank you Benoy. > TestKerberosAuthenticator fails > --- > > Key: HADOOP-8383 > URL: https://issues.apache.org/jira/browse/HADOOP-8383 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 0.22.0 >Reporter: Benoy Antony >Assignee: Benoy Antony >Priority: Minor > Fix For: 0.22.1 > > Attachments: ignore-kerberos-authenticator-test.patch > > > TestKerberosAuthenticator fails when executed on Jenkins > These failures are for tests supposed to be ignored. (annotated with > @Ignore) . But Jenkins executes and reports failures for these tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8381) Substitute _HOST with hostname for HTTP principals
[ https://issues.apache.org/jira/browse/HADOOP-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-8381. - Resolution: Fixed Hadoop Flags: Reviewed +1 I just committed this to branch 0.22.1. Thank you Benoy. > Substitute _HOST with hostname for HTTP principals > > > Key: HADOOP-8381 > URL: https://issues.apache.org/jira/browse/HADOOP-8381 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Affects Versions: 0.22.0 >Reporter: Benoy Antony >Assignee: Benoy Antony >Priority: Minor > Fix For: 0.22.1 > > Attachments: HOST-substitution-spnego.patch, > HOST-substitution-spnego.patch > > > SPNEGO based Web Authentication uses HTTP/fqdn@REALM as the kerberos > principal for each host. > Since it is difficult to modify the config for each host, a substitution > feature where _HOST gets replaced by fqdn is implemented. > The task is to provide similar feature for the kerberos principals used for > SPNEGO principals -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7338) LocalDirAllocator improvements for MR-2178
[ https://issues.apache.org/jira/browse/HADOOP-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-7338. - Resolution: Fixed Hadoop Flags: Reviewed +1 I just committed this to branch 0.22.1. Thank you Benoy. > LocalDirAllocator improvements for MR-2178 > -- > > Key: HADOOP-7338 > URL: https://issues.apache.org/jira/browse/HADOOP-7338 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Benoy Antony > Fix For: 0.22.1 > > Attachments: hadoop-7338-022.patch, hadoop-7338.patch, hadoop-7338.txt > > > These are some changes that come from the following yahoo-merge commit, > necessary for MAPREDUCE-2178 to build. > commit 446e41f2424ddc866770cda677450c18feeb597f > Author: Chris Douglas > Date: Sun Oct 3 02:00:15 2010 -0700 > > Apply y201 to trunk > > git-svn-id: > https://svn.apache.org/repos/asf/hadoop/common/branches/yahoo-merge@1079151 > 13f79535-47bb-0310-9956-ffa450edef68 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7457) Remove out-of-date Chinese language documentation
[ https://issues.apache.org/jira/browse/HADOOP-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-7457. - Resolution: Fixed I applied Jacob's patch and deleted docs/cn in 0.22 branch. > Remove out-of-date Chinese language documentation > - > > Key: HADOOP-7457 > URL: https://issues.apache.org/jira/browse/HADOOP-7457 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 0.21.0 >Reporter: Jakob Homan >Assignee: Jakob Homan >Priority: Blocker > Fix For: 0.22.0, 0.23.0 > > Attachments: HADOOP-7457.patch > > > The Chinese language documentation haven't been updated (other than copyright > years and svn moves) since their original contribution several years ago. > Worse than no docs is out-of-date, wrong docs. We should delete them from > the source tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-7457) Remove out-of-date Chinese language documentation
[ https://issues.apache.org/jira/browse/HADOOP-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reopened HADOOP-7457: - Reopening this to port changes to 0.22 branch. Currently common build fails because of that. See HADOOP-7394 and HADOOP-7303. > Remove out-of-date Chinese language documentation > - > > Key: HADOOP-7457 > URL: https://issues.apache.org/jira/browse/HADOOP-7457 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 0.21.0 >Reporter: Jakob Homan >Assignee: Jakob Homan > Fix For: 0.23.0 > > Attachments: HADOOP-7457.patch > > > The Chinese language documentation haven't been updated (other than copyright > years and svn moves) since their original contribution several years ago. > Worse than no docs is out-of-date, wrong docs. We should delete them from > the source tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-7568) SequenceFile should not print into stdout
SequenceFile should not print into stdout
-
Key: HADOOP-7568
URL: https://issues.apache.org/jira/browse/HADOOP-7568
Project: Hadoop Common
Issue Type: Bug
Reporter: Konstantin Shvachko
The following line in {{SequenceFile.Reader.initialize()}} should be removed:
{code}
System.out.println("Setting end to " + end);
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-7311) Port remaining metrics v1 from trunk to branch-0.20-security
Port remaining metrics v1 from trunk to branch-0.20-security Key: HADOOP-7311 URL: https://issues.apache.org/jira/browse/HADOOP-7311 Project: Hadoop Common Issue Type: Bug Components: metrics Affects Versions: 0.20.203.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Fix For: 0.20.204.0 HADOOP-7190 added metrics packages/classes. This is a port from trunk to make them actually work for the security branch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HADOOP-7155) Move ipc.Server#CURRENT_VERSION constant to VersionedProtocol.
Move ipc.Server#CURRENT_VERSION constant to VersionedProtocol. -- Key: HADOOP-7155 URL: https://issues.apache.org/jira/browse/HADOOP-7155 Project: Hadoop Common Issue Type: Improvement Affects Versions: 0.20.0 Reporter: Konstantin Shvachko Constant o.a.h.ipc.Server#CURRENT_VERSION defines RPC version. It should be tied to the VersionedProtocol interface rather than the ipc.Server implementation. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-7033) getUserToGroupsMappingService is not thread safe
[
https://issues.apache.org/jira/browse/HADOOP-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko resolved HADOOP-7033.
-
Resolution: Duplicate
Fix Version/s: 0.22.0
Closing as duplicate of HADOOP-7046.
> getUserToGroupsMappingService is not thread safe
>
>
> Key: HADOOP-7033
> URL: https://issues.apache.org/jira/browse/HADOOP-7033
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Affects Versions: 0.22.0
>Reporter: Eli Collins
>Assignee: Po Cheung
> Fix For: 0.22.0
>
>
> test-patch on trunk reports the following findbugs warning:
> {noformat}
> Incorrect lazy initialization of static field
> org.apache.hadoop.security.Groups.GROUPS in
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Configuration)
> Bug type LI_LAZY_INIT_STATIC (click for details)
> In class org.apache.hadoop.security.Groups
> In method
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Configuration)
> On field org.apache.hadoop.security.Groups.GROUPS
> At Groups.java:[lines 138-142]
> {noformat}
> The relevant code:
> {noformat}
> public static Groups getUserToGroupsMappingService(Configuration conf) {
> if(GROUPS == null) {
> if(LOG.isDebugEnabled()) {
> LOG.debug(" Creating new Groups object");
> }
> GROUPS = new Groups(conf);
> }
> return GROUPS;
> }
> {noformat}
> This is problematic if getUserToGroupsMappingService can accessed
> simultaneously by multiple threads.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-6949) Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting
[ https://issues.apache.org/jira/browse/HADOOP-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reopened HADOOP-6949: - This seems to have higher impact on performance than previously reported. I see many VersionProtocol methods use arrays as their input or output parameters, which can substantially benefit from this optimization. See discussion started in HDFS-1583. > Reduces RPC packet size for primitive arrays, especially long[], which is > used at block reporting > - > > Key: HADOOP-6949 > URL: https://issues.apache.org/jira/browse/HADOOP-6949 > Project: Hadoop Common > Issue Type: Improvement > Components: io >Reporter: Navis >Priority: Trivial > Attachments: ObjectWritable.diff > > Original Estimate: 0.17h > Remaining Estimate: 0.17h > > Current implementation of oah.io.ObjectWritable marshals primitive array > types as general object array ; array type string + array length + (element > type string + value)*n > It would not be needed to specify each element types for primitive arrays. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-7102) Remove "fs.ramfs.impl" field from core-deafult.xml
Remove "fs.ramfs.impl" field from core-deafult.xml -- Key: HADOOP-7102 URL: https://issues.apache.org/jira/browse/HADOOP-7102 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Fix For: 0.22.0 "fs.ramfs.impl" used to be configuration parameter for InMemoryFileSystem, which was deprecated in 0.18 (HADOOP-3501) and removed in 0.21 (HADOOP-3501). Configuration should have been cleaned up at the time. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6718) Client does not close connection when an exception happens during SASL negotiation
[ https://issues.apache.org/jira/browse/HADOOP-6718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-6718. - Resolution: Duplicate Incorporated in HADOOP-6706 for 0.22. > Client does not close connection when an exception happens during SASL > negotiation > -- > > Key: HADOOP-6718 > URL: https://issues.apache.org/jira/browse/HADOOP-6718 > Project: Hadoop Common > Issue Type: Bug > Components: ipc >Affects Versions: 0.22.0 >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.22.0 > > Attachments: 6718-bp20.patch > > > setupSaslConnection in the RPC client might fail to successfully set up a > sasl connection (e.g. if the principal is wrongly configured). It throws an > exception back to the caller (setupIOstreams). setupIOstreams marks the > connection as closed but it doesn't really close the socket. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6872) ChecksumFs#listStatus should filter out .crc files
[ https://issues.apache.org/jira/browse/HADOOP-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-6872. - Resolution: Duplicate Fixed as a part of HADOOP-6906. > ChecksumFs#listStatus should filter out .crc files > -- > > Key: HADOOP-6872 > URL: https://issues.apache.org/jira/browse/HADOOP-6872 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.21.0 >Reporter: Hairong Kuang > Fix For: 0.22.0 > > > Currently ChecksumFs#listStatus listing not only regular files but also .crc > files. Crc files should be filtered out. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6970) SecurityAuth.audit should be generated under /build
SecurityAuth.audit should be generated under /build --- Key: HADOOP-6970 URL: https://issues.apache.org/jira/browse/HADOOP-6970 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 0.22.0 Reporter: Konstantin Shvachko Fix For: 0.22.0 SecurityAuth.audit is generated under currently root project directory whenever I run anything, and is not being cleaned up by the clean target. It should be created under build directory instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6969) CHANGES.txt does not reflect the release of version 0.21.0.
CHANGES.txt does not reflect the release of version 0.21.0. --- Key: HADOOP-6969 URL: https://issues.apache.org/jira/browse/HADOOP-6969 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.21.0 Environment: CHANGES.txt should show the release date for 0.21.0 and include section for for 0.21.1 - Unreleased. Latest changes, that did not make into 0.21.0, should be moved under 0.21.1 section. Reporter: Konstantin Shvachko Fix For: 0.21.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6756) Clean up and add documentation for configuration keys in CommonConfigurationKeys.java
[ https://issues.apache.org/jira/browse/HADOOP-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-6756. - Hadoop Flags: [Reviewed] Fix Version/s: (was: 0.20.3) Resolution: Fixed I just committed this. Thank you Erik. > Clean up and add documentation for configuration keys in > CommonConfigurationKeys.java > - > > Key: HADOOP-6756 > URL: https://issues.apache.org/jira/browse/HADOOP-6756 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 0.20.3, 0.22.0 >Reporter: Erik Steffl >Assignee: Erik Steffl > Fix For: 0.22.0 > > Attachments: HADOOP-6756-0.22.0-1.patch, HADOOP-6756-0.22.0-2.patch, > HADOOP-6756-0.22.0-3.patch, HADOOP-6756-0.22.0.patch, > jira.HADOOP-6756-0.20-1-FS_DEFAULT_NAME_KEY.patch, > jira.HADOOP-6756-0.20-1.patch, jira.HADOOP-6756-0.20.patch > > > Configuration keys in CommonConfigurationKeys.java should be cleaned up and > documented (javadoc comments, appropriate *-default.xml descriptions). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6074) TestDFSIO does not use configuration properly.
[ https://issues.apache.org/jira/browse/HADOOP-6074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HADOOP-6074. - Resolution: Duplicate This has been fixed by HDFS-459. > TestDFSIO does not use configuration properly. > -- > > Key: HADOOP-6074 > URL: https://issues.apache.org/jira/browse/HADOOP-6074 > Project: Hadoop Common > Issue Type: Bug > Components: benchmarks >Reporter: Raghu Angadi >Assignee: Raghu Angadi >Priority: Minor > Fix For: 0.21.0 > > Attachments: HADOOP-6074.patch > > > A part of TestDFSIO uses default conf instead of configuration passed in by > MR framework. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6637) Benchmark overhead of RPC session establishment
Benchmark overhead of RPC session establishment Key: HADOOP-6637 URL: https://issues.apache.org/jira/browse/HADOOP-6637 Project: Hadoop Common Issue Type: Test Components: benchmarks Affects Versions: 0.20.2 Reporter: Konstantin Shvachko Fix For: 0.20.3 Measure the latency of RPC session establishments through three mechanisms: # simple - no auth # kerberos authentication # delegation token authentication -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6626) NPE in TestIPC with kerberos
NPE in TestIPC with kerberos
Key: HADOOP-6626
URL: https://issues.apache.org/jira/browse/HADOOP-6626
Project: Hadoop Common
Issue Type: Bug
Components: ipc
Affects Versions: 0.22.0
Reporter: Konstantin Shvachko
Fix For: 0.22.0
Running TestIPC with {{hadoop.security.authentication=kerberos}}, but without
any actual Kerberos around results in NPE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6485) Trash fails on Windows
Trash fails on Windows -- Key: HADOOP-6485 URL: https://issues.apache.org/jira/browse/HADOOP-6485 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.20.0 Reporter: Konstantin Shvachko Fix For: 0.20.0 Using local file system on windows Trash tries to move file "file:/C:/tmp/testTrash/foo" to "file:/C:/Documents and Settings/shv/.Trash/Current/C:/tmp/testTrash/foo" with "C:" in the middle. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6243) NPE in handling deprecated configuration keys.
NPE in handling deprecated configuration keys.
--
Key: HADOOP-6243
URL: https://issues.apache.org/jira/browse/HADOOP-6243
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 0.21.0
Reporter: Konstantin Shvachko
Fix For: 0.21.0
I run TestFileCreation in Eclipse and get a NullPointerException. Debugging
shows that {{Configuration.populateDeprecationMapping()}} sets
{{Configuration.properties}} to null.
This code was introduced in HADOOP-6105.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HADOOP-6123) hdfs script does not work after project split.
hdfs script does not work after project split. -- Key: HADOOP-6123 URL: https://issues.apache.org/jira/browse/HADOOP-6123 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Fix For: 0.21.0 There are problems running hdfs script from common. # Usage message for hdfs does not have "fs" option anymore. # There seem to be an undocumented option "dfs", but using it throws NoClassDefFoundError or ClassNotFoundException. # Same with other options e.g. name-node. May I am missing something here. How do we call ls these days? Do we need to move hdfs script to hdfs project. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6080) Handling of Trash with quota
[
https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko resolved HADOOP-6080.
-
Resolution: Fixed
Fix Version/s: 0.20.1
I just committed this. Thank you Jakob.
> Handling of Trash with quota
> -
>
> Key: HADOOP-6080
> URL: https://issues.apache.org/jira/browse/HADOOP-6080
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
>Reporter: Koji Noguchi
>Assignee: Jakob Homan
> Fix For: 0.20.1
>
> Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch,
> javac_warnings_diff.txt
>
>
> Currently with quota turned on, user cannot call '-rmr' on large directory
> that causes over quota.
> {noformat}
> [knoguchi src]$ hadoop dfs -rmr /tmp/net2
> rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
> [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
> mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of
> /user/knoguchi is exceeded: namespace
> quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
> {noformat}
> Besides from error message being unfriendly, how should this be handled?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
