[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969944#comment-14969944
]
Hadoop QA commented on HDFS-9241:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 24m 53s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 8m 38s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 42s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 27s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 47s | The applied patch generated
17 new checkstyle issues (total was 104, now 120). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 42s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 36s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 44m 3s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 31s | Tests passed in
hadoop-hdfs-client. |
| | | 102m 12s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestHFlush |
| | hadoop.hdfs.TestModTime |
| | hadoop.hdfs.TestRemoteBlockReader2 |
| | hadoop.hdfs.TestReplication |
| | hadoop.hdfs.qjournal.client.TestQJMWithFaults |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
| | hadoop.hdfs.TestAbandonBlock |
| | hadoop.hdfs.TestSetTimes |
| | hadoop.hdfs.TestDFSFinalize |
| | hadoop.hdfs.server.blockmanagement.TestNodeCount |
| | hadoop.hdfs.TestAppendDifferentChecksum |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.TestFileCreationClient |
| | hadoop.hdfs.TestDefaultNameNodePort |
| | hadoop.hdfs.web.TestWebHDFS |
| | hadoop.hdfs.TestHdfsAdmin |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.TestReadWhileWriting |
| | hadoop.hdfs.TestFileCreation |
| | hadoop.hdfs.TestRead |
| | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean |
| | hadoop.hdfs.crypto.TestHdfsCryptoStreams |
| | hadoop.hdfs.qjournal.TestSecureNNWithQJM |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestWriteConfigurationToDFS |
| | hadoop.hdfs.TestParallelShortCircuitReadNoChecksum |
| | hadoop.hdfs.util.TestBestEffortLongFile |
| | hadoop.hdfs.qjournal.client.TestQuorumJournalManager |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.TestEncryptionZonesWithKMS |
| | hadoop.hdfs.TestFileAppend |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.TestFileCorruption |
| | hadoop.hdfs.TestQuota |
| | hadoop.hdfs.TestRemoteBlockReader |
| | hadoop.hdfs.TestDFSShellGenericOptions |
| | hadoop.hdfs.TestHDFSServerPorts |
| | hadoop.hdfs.qjournal.server.TestJournalNode |
| | hadoop.hdfs.TestFileStatusWithECPolicy |
| | hadoop.hdfs.TestLocalDFS |
| | hadoop.hdfs.TestLease |
| | hadoop.hdfs.TestIsMethodSupported |
| | hadoop.hdfs.TestWriteRead |
| | hadoop.hdfs.TestFileAppend2 |
| | hadoop.hdfs.TestSetrepIncreasing |
| | hadoop.hdfs.TestDatanodeDeath |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.TestBlockReaderFactory |
| | hadoop.hdfs.qjournal.client.TestEpochsAreUnique |
| | hadoop.hdfs.TestClose |
| | hadoop.hdfs.TestDeprecatedKeys |
| | hadoop.hdfs.TestDataTransferKeepalive |
| | hadoop.hdfs.TestDatanodeLayoutUpgrade |
| | hadoop.hdfs.TestDFSUtil |
| | hadoop.hdfs.TestPersistBlocks |
| | hadoop.hdfs.TestErasureCodingPolicies |
| | hadoop.hdfs.TestHDFSTrash |
| | hadoop.hdfs.TestFileCreationEmpty |
| | hadoop.hdfs.TestFileCreationDelete |
| | hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs |
| | hadoop.hdfs.TestListFilesInDFS |
| | hadoop.hdfs.TestDatanodeConfig |
| | hadoop.hdfs.TestEncryptionZonesWithHA |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead |
| | hadoop.hdfs.TestDFSStorageStateRecovery |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.TestDisableConnCache |
| | hadoop.hdfs.TestPipelines |
| |
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970453#comment-14970453
]
Hadoop QA commented on HDFS-9241:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 21m 22s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 9m 0s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 45s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 54s | The applied patch generated 1
new checkstyle issues (total was 104, now 104). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 53s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 56s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 44s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 54m 47s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 34s | Tests passed in
hadoop-hdfs-client. |
| | | 111m 59s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.fs.contract.hdfs.TestHDFSContractMkdir |
| | hadoop.hdfs.qjournal.TestMiniJournalCluster |
| | hadoop.cli.TestDeleteCLI |
| | hadoop.hdfs.TestFileLengthOnClusterRestart |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory |
| | hadoop.cli.TestHDFSCLI |
| | hadoop.hdfs.TestReplaceDatanodeOnFailure |
| | hadoop.hdfs.TestDFSStorageStateRecovery |
| | hadoop.fs.contract.hdfs.TestHDFSContractRename |
| | hadoop.cli.TestCacheAdminCLI |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | hadoop.fs.loadGenerator.TestLoadGenerator |
| | hadoop.fs.TestFcHdfsSetUMask |
| | hadoop.hdfs.crypto.TestHdfsCryptoStreams |
| | hadoop.fs.viewfs.TestViewFsFileStatusHdfs |
| | hadoop.tracing.TestTracingShortCircuitLocalRead |
| | hadoop.tracing.TestTracing |
| | hadoop.hdfs.TestDFSStripedInputStream |
| | hadoop.fs.TestWebHdfsFileContextMainOperations |
| | hadoop.hdfs.TestParallelShortCircuitReadNoChecksum |
| | hadoop.hdfs.TestParallelShortCircuitLegacyRead |
| | hadoop.net.TestNetworkTopology |
| | hadoop.fs.TestFcHdfsCreateMkdir |
| | hadoop.fs.TestFcHdfsPermission |
| | hadoop.hdfs.TestQuota |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.TestDFSStartupVersions |
| | hadoop.fs.viewfs.TestViewFsWithXAttrs |
| | hadoop.cli.TestErasureCodingCLI |
| | hadoop.tools.TestJMXGet |
| | hadoop.TestGenericRefresh |
| | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.TestDataTransferKeepalive |
| | hadoop.fs.TestSWebHdfsFileContextMainOperations |
| | hadoop.fs.contract.hdfs.TestHDFSContractDelete |
| | hadoop.fs.contract.hdfs.TestHDFSContractSetTimes |
| | hadoop.hdfs.TestConnCache |
| | hadoop.hdfs.TestFileAppend |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.TestDisableConnCache |
| | hadoop.cli.TestXAttrCLI |
| | hadoop.fs.TestUrlStreamHandler |
| | hadoop.tools.TestTools |
| | hadoop.fs.TestEnhancedByteBufferAccess |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.fs.viewfs.TestViewFileSystemWithXAttrs |
| | hadoop.fs.TestHDFSFileContextMainOperations |
| | hadoop.hdfs.qjournal.client.TestEpochsAreUnique |
| | hadoop.hdfs.TestExternalBlockReader |
| | hadoop.fs.contract.hdfs.TestHDFSContractAppend |
| | hadoop.hdfs.TestDFSStripedOutputStream |
| | hadoop.hdfs.TestHdfsAdmin |
| | hadoop.fs.contract.hdfs.TestHDFSContractOpen |
| | hadoop.hdfs.TestFileCreation |
| | hadoop.hdfs.TestClientReportBadBlock |
| | hadoop.tracing.TestTraceAdmin |
| | hadoop.hdfs.TestAbandonBlock |
| | hadoop.hdfs.TestDFSInputStream |
| | hadoop.hdfs.TestWriteBlockGetsBlockLengthHint |
| | hadoop.fs.viewfs.TestViewFsHdfs |
| | hadoop.fs.contract.hdfs.TestHDFSContractConcat |
| | hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot |
| | hadoop.cli.TestCryptoAdminCLI |
| | hadoop.hdfs.TestFileCreationDelete |
| | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean |
| |
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967597#comment-14967597
]
Haohui Mai commented on HDFS-9241:
--
{code}
+ String DFS_NAMENODE_BACKUP_ADDRESS_KEY = "dfs.namenode.backup.address";
+ String DFS_NAMENODE_BACKUP_HTTP_ADDRESS_KEY =
"dfs.namenode.backup.http-address";
...
{code}
It makes sense to wrap all these strings into an interface to explicitly state
that they are deprecated keys. For example.
{code}
/**
* Some comments here
**/
interface DeprecatedKeys {
String ...
}
{code}
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch, HDFS-9241.001.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968008#comment-14968008
]
Hadoop QA commented on HDFS-9241:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 20m 30s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 8m 12s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 32s | The applied patch generated
108 new checkstyle issues (total was 105, now 212). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 31s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 0m 21s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 29s | Tests passed in
hadoop-hdfs-client. |
| | | 53m 3s | |
\\
\\
|| Reason || Tests ||
| Failed build | hadoop-hdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12767860/HDFS-9241.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 25f8f80 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13114/console |
This message was automatically generated.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch, HDFS-9241.001.patch,
> HDFS-9241.002.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966609#comment-14966609
]
Steve Loughran commented on HDFS-9241:
--
I like this. I'll still have to move my code off DFSConfigKeys, which means
some inlining of constants, but at least the critical issue, getting those
hdfs-site and hdfs-default XML resources loaded will be addressed. That's the
thing that would have really caused problems.
I'll leave for others to give a final vote, but I'm personally +1 here
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch, HDFS-9241.001.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966416#comment-14966416
]
Hadoop QA commented on HDFS-9241:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 20m 3s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 57s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 29s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 29s | The applied patch generated
91 new checkstyle issues (total was 104, now 194). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 34s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 50m 18s | Tests passed in hadoop-hdfs.
|
| {color:green}+1{color} | hdfs tests | 0m 32s | Tests passed in
hadoop-hdfs-client. |
| | | 102m 18s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12767728/HDFS-9241.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0c4af0f |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13102/console |
This message was automatically generated.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch, HDFS-9241.001.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965550#comment-14965550
]
Colin Patrick McCabe commented on HDFS-9241:
Protobuf is used in RPCv9. The client must speak that. Guava is used
extensively in the client as well. You will need commons-logging, and all the
other logging stuff too. commons-codec and commons-io will be needed to
decompress data. Those are just the ones I can think of off the top of my head.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965593#comment-14965593
]
Steve Loughran commented on HDFS-9241:
--
Netty? client side? I thought changes in ZK's dependencies there was what was
breaking hadoop-hdfs builds on bigtop (HADOOP-12415). hdfs-client shouldn't be
needing netty. There's no jersey, none of its bits, and you could probably cull
curator.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963045#comment-14963045
]
Steve Loughran commented on HDFS-9241:
--
I see your point, but I will note that it loses part of the benefit of the
whole idea of a leaner client "you miss out all the server-side dependencies"
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964044#comment-14964044
]
Colin Patrick McCabe commented on HDFS-9241:
bq. I see your point, but I will note that it loses part of the benefit of the
whole idea of a leaner client "you miss out all the server-side dependencies"
As we commented when the project started, the client is not "thin"-- most of
the problematic dependencies like Jackson, Guava, Jetty, Jersey, Netty, Xerces,
Protobuf are included in the client anyway. So the client/server split
provides no meaningful dependency isolation with or without this change.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963652#comment-14963652
]
Mingliang Liu commented on HDFS-9241:
-
I think you two [~wheat9] and [~steve_l] have made good points. I'll update the
patch soon.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964272#comment-14964272
]
Haohui Mai commented on HDFS-9241:
--
bq. most of the problematic dependencies like Jackson, Guava, Jetty, Jersey,
Netty, Xerces, Protobuf are included in the client anyway
I found that the above statement inaccurate. Many of the above dependency comes
as transitive dependency from hadoop-common. They are not needed from
hadoop-hdfs-client. They can be excluded as what have been done in hadoop
client. Just to give an example, we cleaned up the implementation of log4j in
the module (HDFS-6564). I don't see how this can be done correctly without the
breakdown.
As the refactor continues to break down hadoop-common into server / client
modules, I anticipate that this is a non-issue. The shadowing approach is
complementary. However, having a client that shadows 2000 classes that it
doesn't need that it happens to work at a first glance but it doesn't seems
elegant and responsible in the long term.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963625#comment-14963625
]
Haohui Mai commented on HDFS-9241:
--
Totally agree. Let's go ahead with your proposals :-)
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962773#comment-14962773
]
Haohui Mai commented on HDFS-9241:
--
Though I don't think this is a strictly incompatible change as the applications
won't break if they still depend on hadoop-hdfs, I see a lot of values to allow
the applications to just change their dependency to hadoop-hdfs-client and
without changing the code.
I think the proposal makes a lot of sense. +1 on the proposal. [~liuml07], do
you agree?
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961842#comment-14961842
]
Steve Loughran commented on HDFS-9241:
--
to clarify then
# the thin client doesn't have a backwards compatible way to force load
hdfs-site
# the "two lines of code" proposed as a workaround is in fact package private
# the reason for introducing this change is because there are some deprecated
tags
I don't want to come over all [~aw] here but this isn't justification for
making things incompatible. And while, yes, I can include the hdfs all JAR,
that misses the purpose of the hdfs-client package: to produce a leaner
client-side only package.
We've long hand a problem in HDFS, that whereas Yarn's {{YarnConfiguration}},
and indeed {{JobConfiguration}} have been public with stable string constants
designed for public consumption, the sole set of string constants in HDFS have
been considered private, free to change on a whim. Either we downstream
developers end up importing and something which has a history of being broken
(HDFS-6418) on the grounds that "people downstream should have cut and paste
strings into their source". At least when people use {{DFSConfigKeys}} values,
you can use the IDE to find the load points.
I propose
# accepting that yes, there are deprecated constants in {{DFSConfigKeys}}, but
they are used in client apps
# moving it and the HdfsConfiguration class into hdfs-client
It's not going to add new dependencies, and it will retain compatibility.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961839#comment-14961839
]
Steve Loughran commented on HDFS-9241:
--
bq. My answer is no – the current implementation has a class
HdfsConfigurationLoader to load the configurations that serves the original
purposes of HdfsConfiguration on the client side.
You've just created an incompatible change on branch-2. I've hit this problem
in Slider, which we build against 2.6, only now find it doesn't. Others may
have similar problem.
bq. The reason is that HdfsConfiguration are used by both the client and the
server side. It contains deprecated keys for the server side, which IMO should
not be exposed to the clients at all.
welll, they are tagged as deprecated. Again, they may get used.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960785#comment-14960785
]
Steve Loughran commented on HDFS-9241:
--
One other thing to consider is "would we expect thin clients to ever
instantiate this class?". If so, should it be in that JAR.
until now, creating it has been the way to force hdfs-site in, just as creating
a {{YarnConfiguration()}} forced that in. After hitting problems with race
conditions in UGI init, I now load all of these on startup. Should this be
necessary?
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961557#comment-14961557
]
Mingliang Liu commented on HDFS-9241:
-
{quote}
Old applications can still depend on hadoop-hdfs and nothing will break.
However, the application might need to change a couple lines of code if it only
wants to depend on hadoop-hdfs-client.
{quote}
It makes sense to me. Do you think we need to make {{HdfsConfigurationLoader}}
public so that code depending on {{hadoop-hdfs-client}} is able to load the
default resource forcefully (in case)?
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960983#comment-14960983
]
Haohui Mai commented on HDFS-9241:
--
bq. One other thing to consider is "would we expect thin clients to ever
instantiate this class?". If so, should it be in that JAR.
My answer is no -- the current implementation has a class
{{HdfsConfigurationLoader}} to load the configurations that serves the original
purposes of {{HdfsConfiguration}} on the client side.
The reason is that {{HdfsConfiguration}} are used by both the client and the
server side. It contains deprecated keys for the server side, which IMO should
not be exposed to the clients at all.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960987#comment-14960987
]
Haohui Mai commented on HDFS-9241:
--
bq. the changes for the hdfs client classpath make instantiating
HdfsConfiguration from the client impossible; it only lives server side. This
breaks any app which creates one.
I'm trying to understand the use cases of applications creating a
{{HdfsConfiguration}} instance. Is it because that the apps need a way to force
the hdfs configurations to be loaded?
Old applications can still depend on {{hadoop-hdfs}} and nothing will break.
However, the application might need to change a couple lines of code if it only
wants to depend on {{hadoop-hdfs-client}}. Thoughts?
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957711#comment-14957711
]
Mingliang Liu commented on HDFS-9241:
-
Thanks for reporting this [~steve_l]. I'll update the patch accordingly.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957705#comment-14957705
]
Steve Loughran commented on HDFS-9241:
--
lets see what happens in the discussion before voting on this
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9241) HDFS clients can't construct HdfsConfiguration instances
[
https://issues.apache.org/jira/browse/HDFS-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957912#comment-14957912
]
Hadoop QA commented on HDFS-9241:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 20m 13s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 10m 15s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 13m 41s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 29s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 2m 0s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 45s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | client tests | 0m 17s | Tests passed in
hadoop-client. |
| | | 47m 43s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766630/HDFS-9241.000.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / 3d50855 |
| hadoop-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12991/artifact/patchprocess/testrun_hadoop-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12991/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12991/console |
This message was automatically generated.
> HDFS clients can't construct HdfsConfiguration instances
>
>
> Key: HDFS-9241
> URL: https://issues.apache.org/jira/browse/HDFS-9241
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Reporter: Steve Loughran
>Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> the changes for the hdfs client classpath make instantiating
> {{HdfsConfiguration}} from the client impossible; it only lives server side.
> This breaks any app which creates one.
> I know people will look at the {{@Private}} tag and say "don't do that then",
> but it's worth considering precisely why I, at least, do this: it's the only
> way to guarantee that the hdfs-default and hdfs-site resources get on the
> classpath, including all the security settings. It's precisely the use case
> which {{HdfsConfigurationLoader.init();}} offers internally to the hdfs code.
> What am I meant to do now?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
