[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410393#comment-16410393 ] Hudson commented on HDFS-9538: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13869 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13869/]) HDFS-9538. libhdfs: load configuration from files. Contributed by Bob (james.clampffer: rev 6c80d02a03deadf15449cf32f728f0a9c58e3d89) * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/configuration_loader.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/configuration_loader_impl.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_configuration_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/configuration_test.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/hdfs_configuration.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/hdfs_configuration.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/configuration.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/configuration_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/configuration_loader.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/configuration.h > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen >Priority: Major > Attachments: HDFS-9538.HDFS-8707.003.patch, > HDFS-9538.HDFS-9537.000.patch, HDFS-9538.HDFS-9537.001.patch, > HDFS-9538.HDFS-9537.002.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058272#comment-15058272 ] James Clampffer commented on HDFS-9538: --- Committed to HDFS-8707. Thanks for the contribution [~bobthansen]! > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > Attachments: HDFS-9538.HDFS-8707.003.patch, > HDFS-9538.HDFS-9537.000.patch, HDFS-9538.HDFS-9537.001.patch, > HDFS-9538.HDFS-9537.002.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[
https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056757#comment-15056757
]
Hadoop QA commented on HDFS-9538:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 52s
{color} | {color:green} HDFS-8707 passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 53s
{color} | {color:green} HDFS-8707 passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 16s
{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 58s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 16s {color}
| {color:red} hadoop-hdfs-native-client in the patch failed with JDK v1.8.0_66.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 4m 24s {color}
| {color:red} hadoop-hdfs-native-client in the patch failed with JDK v1.7.0_91.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12777575/HDFS-9538.HDFS-8707.003.patch
|
| JIRA Issue | HDFS-9538 |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 44d0a850080f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-8707 / 4a0eea7 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/13868/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.8.0_66.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/13868/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_91.txt
|
| JDK v1.7.0_91 Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/13868/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Max memory used | 76MB |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13868/console |
This message was automatically generated.
> libhdfs++: load configuration from files
>
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056713#comment-15056713 ] James Clampffer commented on HDFS-9538: --- "To be able to re-use the calculations for Java classpaths as a search path for the config files." Sounds good, makes sense. "I took out the "file_exists" call, and just try to read from the file. If we can't then we go on as if it doesn't exist." Nice, that works. "We now check the results and just fail the test if something goes wrong." Sounds good to me. +1, I'll commit sometime tomorrow unless someone else spots an issue. > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > Attachments: HDFS-9538.HDFS-8707.003.patch, > HDFS-9538.HDFS-9537.000.patch, HDFS-9538.HDFS-9537.001.patch, > HDFS-9538.HDFS-9537.002.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[
https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056661#comment-15056661
]
Hadoop QA commented on HDFS-9538:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HDFS-9538 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12777567/HDFS-9538.HDFS-9537.002.patch
|
| JIRA Issue | HDFS-9538 |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13866/console |
This message was automatically generated.
> libhdfs++: load configuration from files
>
>
> Key: HDFS-9538
> URL: https://issues.apache.org/jira/browse/HDFS-9538
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Bob Hansen
>Assignee: Bob Hansen
> Attachments: HDFS-9538.HDFS-9537.000.patch,
> HDFS-9538.HDFS-9537.001.patch, HDFS-9538.HDFS-9537.002.patch
>
>
> One goal of the Configuration classes are to allow the consumers of the
> libhdfs++ library to deploy client applications into hadoop edge nodes and
> have them pick up the Hadoop configuration that has been deployed there.
> Note that we also need to support the use case where the consumer application
> will manage Hadoop configuration files itself, or will handle all
> configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily
> construct an instance that will communicate with the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[
https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056658#comment-15056658
]
Bob Hansen commented on HDFS-9538:
--
New patch to address [~James Clampffer]'s comments.
bq. -It looks like ConfigurationLoader::SetSearchPath is breaking up the path
into subpaths of the full path. What's the intended use case for this?
To be able to re-use the calculations for Java classpaths as a search path for
the config files.
{quote}
-Should file_exists return false if the process doesn't have read permissions
for the file? I'd vote yes
-Regarding the "Can we tell if it's a (transitive) symlink to a regular file?"
comment in file_exists. I think a symlink that resolves to a file that would
return true if used directly with file_exists should also return true. It looks
like it could be done by checking S_ISLNK(my_stat_struct.st_mode) + realpath
recursively with some reasonable depth limit to cover a few levels of
indirection.
{quote}
I took out the "file_exists" call, and just try to read from the file. If we
can't then we go on as if it doesn't exist.
{quote}
-nftw_remove calls perror to handle error conditions. Should this have an
#ifdef in case the user doesn't want things being printed to stderr?
-The return code for the mkdir call in TempDir::Tempdir() is never checked.
Should this have a check that clears out path, or bails if path is already
empty, in case of permissions issues?
{quote}
We now check the results and just fail the test if something goes wrong.
> libhdfs++: load configuration from files
>
>
> Key: HDFS-9538
> URL: https://issues.apache.org/jira/browse/HDFS-9538
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Bob Hansen
>Assignee: Bob Hansen
> Attachments: HDFS-9538.HDFS-9537.000.patch,
> HDFS-9538.HDFS-9537.001.patch, HDFS-9538.HDFS-9537.002.patch
>
>
> One goal of the Configuration classes are to allow the consumers of the
> libhdfs++ library to deploy client applications into hadoop edge nodes and
> have them pick up the Hadoop configuration that has been deployed there.
> Note that we also need to support the use case where the consumer application
> will manage Hadoop configuration files itself, or will handle all
> configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily
> construct an instance that will communicate with the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056421#comment-15056421 ] James Clampffer commented on HDFS-9538: --- A couple more comments: -nftw_remove calls perror to handle error conditions. Should this have an #ifdef in case the user doesn't want things being printed to stderr? -The return code for the mkdir call in TempDir::Tempdir() is never checked. Should this have a check that clears out path, or bails if path is already empty, in case of permissions issues? > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > Attachments: HDFS-9538.HDFS-9537.000.patch, > HDFS-9538.HDFS-9537.001.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[
https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056027#comment-15056027
]
Hadoop QA commented on HDFS-9538:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HDFS-9538 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12777470/HDFS-9538.HDFS-9537.001.patch
|
| JIRA Issue | HDFS-9538 |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13860/console |
This message was automatically generated.
> libhdfs++: load configuration from files
>
>
> Key: HDFS-9538
> URL: https://issues.apache.org/jira/browse/HDFS-9538
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Bob Hansen
>Assignee: Bob Hansen
> Attachments: HDFS-9538.HDFS-9537.000.patch,
> HDFS-9538.HDFS-9537.001.patch
>
>
> One goal of the Configuration classes are to allow the consumers of the
> libhdfs++ library to deploy client applications into hadoop edge nodes and
> have them pick up the Hadoop configuration that has been deployed there.
> Note that we also need to support the use case where the consumer application
> will manage Hadoop configuration files itself, or will handle all
> configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily
> construct an instance that will communicate with the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[
https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053726#comment-15053726
]
Hadoop QA commented on HDFS-9538:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HDFS-9538 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12776950/HDFS-9538.HDFS-9537.000.patch
|
| JIRA Issue | HDFS-9538 |
| Powered by | Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/13839/console |
This message was automatically generated.
> libhdfs++: load configuration from files
>
>
> Key: HDFS-9538
> URL: https://issues.apache.org/jira/browse/HDFS-9538
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Bob Hansen
>Assignee: Bob Hansen
> Attachments: HDFS-9538.HDFS-9537.000.patch
>
>
> One goal of the Configuration classes are to allow the consumers of the
> libhdfs++ library to deploy client applications into hadoop edge nodes and
> have them pick up the Hadoop configuration that has been deployed there.
> Note that we also need to support the use case where the consumer application
> will manage Hadoop configuration files itself, or will handle all
> configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily
> construct an instance that will communicate with the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053555#comment-15053555 ] James Clampffer commented on HDFS-9538: --- Just a little early feedback based on a quick look. Once HDFS-9537 lands I can give more detailed feedback; nothing major jumped out at me on this pass. -It looks like ConfigurationLoader::SetSearchPath is breaking up the path into subpaths of the full path. What's the intended use case for this? -Should file_exists return false if the process doesn't have read permissions for the file? I'd vote yes. -Regarding the "Can we tell if it's a (transitive) symlink to a regular file?" comment in file_exists. I think a symlink that resolves to a file that would return true if used directly with file_exists should also return true. It looks like it could be done by checking S_ISLNK(my_stat_struct.st_mode) + realpath recursively with some reasonable depth limit to cover a few levels of indirection. > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > Attachments: HDFS-9538.HDFS-9537.000.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052057#comment-15052057 ] Bob Hansen commented on HDFS-9538: -- Note: patch applies on top of HDFS-9537.HDFS-8707.001.patch. Will re-submit for CI once 9537 lands, but this can be reviewed now if you'd like. > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > Attachments: HDFS-9538.HDFS-9537.000.patch > > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
[ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051168#comment-15051168 ] Bob Hansen commented on HDFS-9538: -- Proposed design: * Add methods to LoadFromFile and OverlayResourceFile to Configuration and HdfsConfiguration * Have the concept of a search path to separate the concerns of what data is needed (e.g. core-site.xml) vs. where to find it (e.g. HADOOP_CONF_DIR) * Doing environmental substitution (HADOOP_CONF_DIR) will be part of HDFS-9385 * This implementation will look for files in the search paths, but will not search inside of jar files. While the Java implementation _can_ distribute config files in Jar files, is that commonly done in the field? * We will handle implementing the default configuration values in HDFS-9539 > libhdfs++: load configuration from files > > > Key: HDFS-9538 > URL: https://issues.apache.org/jira/browse/HDFS-9538 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Bob Hansen >Assignee: Bob Hansen > > One goal of the Configuration classes are to allow the consumers of the > libhdfs++ library to deploy client applications into hadoop edge nodes and > have them pick up the Hadoop configuration that has been deployed there. > Note that we also need to support the use case where the consumer application > will manage Hadoop configuration files itself, or will handle all > configuration out-of-band. > libhdfs++ should be able to read files that are found in the field and easily > construct an instance that will communicate with the cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
