[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928568#comment-15928568 ] Junping Du commented on HDFS-6200: -- Due to HDFS-11431, I just updated release note here with adding "Please note that hadoop-hdfs-client module could miss class like ConfiguredFailoverProxyProvider. So if a cluster is in HA deployment, we should still use hadoop-hdfs instead." HDFS folks, please check if this is proper notes. Thanks! > Create a separate jar for hdfs-client > - > > Key: HDFS-6200 > URL: https://issues.apache.org/jira/browse/HDFS-6200 > Project: Hadoop HDFS > Issue Type: Improvement > Components: build >Reporter: Haohui Mai >Assignee: Haohui Mai > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, > HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, > HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch > > > Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs > client. As discussed in the hdfs-dev mailing list > (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), > downstream projects are forced to bring in additional dependency in order to > access hdfs. The additional dependency sometimes can be difficult to manage > for projects like Apache Falcon and Apache Oozie. > This jira proposes to create a new project, hadoop-hdfs-cliient, which > contains the client side of the hdfs code. Downstream projects can use this > jar instead of the hadoop-hdfs to avoid unnecessary dependency. > Note that it does not break the compatibility of downstream projects. This is > because old downstream projects implicitly depend on hadoop-hdfs-client > through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15113282#comment-15113282 ] Yongjun Zhang commented on HDFS-6200: - Hi [~wheat9], Thanks for working on this issue. I wonder whether we should mark this jira as incompatible? --Yongjun > Create a separate jar for hdfs-client > - > > Key: HDFS-6200 > URL: https://issues.apache.org/jira/browse/HDFS-6200 > Project: Hadoop HDFS > Issue Type: Improvement > Components: build >Reporter: Haohui Mai >Assignee: Haohui Mai > Fix For: 2.8.0 > > Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, > HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, > HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch > > > Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs > client. As discussed in the hdfs-dev mailing list > (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), > downstream projects are forced to bring in additional dependency in order to > access hdfs. The additional dependency sometimes can be difficult to manage > for projects like Apache Falcon and Apache Oozie. > This jira proposes to create a new project, hadoop-hdfs-cliient, which > contains the client side of the hdfs code. Downstream projects can use this > jar instead of the hadoop-hdfs to avoid unnecessary dependency. > Note that it does not break the compatibility of downstream projects. This is > because old downstream projects implicitly depend on hadoop-hdfs-client > through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957045#comment-14957045
]
Hadoop QA commented on HDFS-6200:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12651333/HDFS-6200.007.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d6c8bad |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12980/console |
This message was automatically generated.
> Create a separate jar for hdfs-client
> -
>
> Key: HDFS-6200
> URL: https://issues.apache.org/jira/browse/HDFS-6200
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: build
>Reporter: Haohui Mai
>Assignee: Haohui Mai
> Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
> HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
> HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
>
>
> Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
> client. As discussed in the hdfs-dev mailing list
> (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
> downstream projects are forced to bring in additional dependency in order to
> access hdfs. The additional dependency sometimes can be difficult to manage
> for projects like Apache Falcon and Apache Oozie.
> This jira proposes to create a new project, hadoop-hdfs-cliient, which
> contains the client side of the hdfs code. Downstream projects can use this
> jar instead of the hadoop-hdfs to avoid unnecessary dependency.
> Note that it does not break the compatibility of downstream projects. This is
> because old downstream projects implicitly depend on hadoop-hdfs-client
> through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619854#comment-14619854
]
Yi Liu commented on HDFS-6200:
--
It's great to have separate jar for hdfs-client.
I have following comments:
*1.* For long term, should we create a project {{hadoop-hdfs-common}}, and move
the common parts which are used by hdfs client and server into that project,
such as protocol, security token related definitions? It's a bit odd to put
those in {{hadoop-hdfs-client}}
*2.* Currently I see we move WebHdfsFileSystem and few others into the
{{hadoop-hdfs-client}}, I think moving the DFSClient and DistributedFileSystem
into {{hadoop-hdfs-client}} is also in the plan?
*3.* I think moving the native hdfs client into this project is also in the
plan?
*4.* Our target of having separate jar for hdfs-client is reducing the jar
dependency, as [~tucu00] said, it can reduce these jars only required by hdfs
server, not sure how many they are after we move all client related into the
project. If there are few, it seems not big sense we do this work. Actually I
think we can indeed reduce many, and I need to verify it.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Components: build
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617621#comment-14617621
]
Hadoop QA commented on HDFS-6200:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12651333/HDFS-6200.007.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c9dd2ca |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11609/console |
This message was automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Components: build
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350369#comment-14350369
]
Sean Busbey commented on HDFS-6200:
---
The dependencies you bring with you are an integral part of the interface you
define for downstream clients. While I agree that it can be a separate subtask,
it has to be considered as part of how you structure the overall approach.
{quote}
Unfortunately the dependency is a real one – the webhdfs server on DN uses
DFSClient to read data from HDFS.
{quote}
Our own internal use of client interfaces isn't the same thing as downstream
application uses. For one, we don't have to worry about what dependencies we
bring with us in the internal case because by definition we're in control of
both the client interface and the place it's being used.
In the approach I'm suggesting the original code for the client would still
live in hadoop-hdfs, so the webhdfs server would be free to use on DFSClient.
If that is unappealing for some reason, perhaps we should structure things with
an internal client artifact. e.g.
{noformat}
hadoop-hdfs -- depends on -- hadoop-hdfs-client-internal
hadoop-hdfs-client -- depends on -- hadoop-hdfs-client-internal
{noformat}
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350711#comment-14350711 ] Sean Busbey commented on HDFS-6200: --- As I mentioned earlier, the dependencies your client artifact brings with it is a defining part of the interface you are exposing downstream applications to. That means we need the ability to manipulate those dependencies, even if we're only going to do so at a later date. Placing hadoop-hdfs-client as a dependency of hadoop-hdfs sets up a relationship that we'll have to adjust in the future if we e.g. decide that shading the third-party dependencies of hadoop-hdfs-client is the way to go. I only mention the internal artifact as an alternative if having DFSClient live in hadoop-hdfs is undesirable. Personally, I think having things stay where they are and using maven to build the client artifact will be the easiest to maintain. However, there might be other mitigating factors I'm not aware of that make breaking the code into a new module desirable. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350684#comment-14350684
]
Haohui Mai commented on HDFS-6200:
--
bq. For one, we don't have to worry about what dependencies we bring with us in
the internal case because by definition we're in control of both the client
interface and the place it's being used.
bq. In the approach I'm suggesting the original code for the client would still
live in hadoop-hdfs, so the webhdfs server would be free to use on DFSClient.
If that is unappealing for some reason, perhaps we should structure things with
an internal client artifact. e.g.
What is the point of creating yet another internal jar if you can simply hide
{{DFSClient}} in local package?
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350783#comment-14350783 ] Haohui Mai commented on HDFS-6200: -- Thanks touch. Just to clarify -- I'm not trashing the classloader solution, I agree that it has its own values on yarn/mr side. I don't see them as competing solutions, they provide values in different use cases. I think we don't need to mix the two issues. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350765#comment-14350765
]
Haohui Mai commented on HDFS-6200:
--
bq. Placing hadoop-hdfs-client as a dependency of hadoop-hdfs sets up a
relationship that we'll have to adjust in the future if we e.g. decide that
shading the third-party dependencies of hadoop-hdfs-client is the way to go.
Don't you agree we need a client jar?
I see you point. This jira, however, is about creating the client jar.
Everything below the client jar is implementation detail. I don't think it need
to be mixed with this jira.
bq. Personally, I think having things stay where they are and using maven to
build the client artifact will be the easiest to maintain
I don't agree. We did that for {{hadoop-client}}, which is available today.
You're more than welcome to contribute and to clean things up. We've been hit
really hard on resolving dependency conflicts in Oozie (which uses tomcat's
classloader), Ranger (depends on different version of jersey-server), Spark
(has a conflicting version of asm). We need a client jar whose dependency can
be carefully and explicitly managed.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350774#comment-14350774 ] Alejandro Abdelnur commented on HDFS-6200: -- Haohui, Doing what hadoop-client wont solve the problems you want to tackle, it will just remove the JARs used on the HDFS server side only. If you just care about those server side dependencies, hadoop-client should be enough and you could exclude YARN/MR artifacts in your dependency. If you want take care of guava, commons-*, etc, etc, you'll need to classloader magic for the filesystem impls, and this should be done in common where the Hadoop FileSystem API lives so all Hadoop FileSystem implementations get this kind of isolation. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350786#comment-14350786 ] Alejandro Abdelnur commented on HDFS-6200: -- Haohui, Could you please list the actual set of dependencies the hdfs-client will carry? Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350995#comment-14350995
]
Haohui Mai commented on HDFS-6200:
--
Here is the list of dependency when I run {{mvn dependency:tree}} in
{{hadoop-hdfs}}:
{noformat}
$ mvn dependency:tree|grep -v :test
...
[INFO] --- maven-dependency-plugin:2.2:tree (default-cli) @ hadoop-hdfs ---
[INFO] org.apache.hadoop:hadoop-hdfs:jar:3.0.0-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-SNAPSHOT:provided
[INFO] | \- jdk.tools:jdk.tools:jar:1.8:system
[INFO] +- org.apache.hadoop:hadoop-auth:jar:3.0.0-SNAPSHOT:provided
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.10:provided
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:provided
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.5:provided (version
managed from 4.2.4)
[INFO] | +-
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:provided
[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:provided
[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:provided
[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:provided
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:provided
[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:provided
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.0.0-SNAPSHOT:provided
[INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:provided
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:provided
[INFO] | +- commons-net:commons-net:jar:3.1:provided
[INFO] | +- commons-collections:commons-collections:jar:3.2.1:provided
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:provided
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:provided
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:provided
[INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided
[INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:provided
[INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:provided
[INFO] | | | \- javax.activation:activation:jar:1.1:provided
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:provided (version
managed from 1.8.3)
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:provided (version
managed from 1.8.3)
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:provided
[INFO] | | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:provided
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:provided
[INFO] | | +- commons-digester:commons-digester:jar:1.8:provided
[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:provided
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:provided
[INFO] | +- org.apache.avro:avro:jar:1.7.4:provided
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:provided
[INFO] | | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:provided
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:provided
[INFO] | +- com.jcraft:jsch:jar:0.1.42:provided
[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:provided
[INFO] | +- org.apache.curator:curator-recipes:jar:2.7.1:provided
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:provided
[INFO] | \- org.tukaani:xz:jar:1.0:provided
[INFO] +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | \- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO] +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO] | \- asm:asm:jar:3.2:compile (version managed from 3.1)
[INFO] +- commons-cli:commons-cli:jar:1.2:compile
[INFO] +- commons-codec:commons-codec:jar:1.4:compile
[INFO] +- commons-io:commons-io:jar:2.4:compile
[INFO] +- commons-lang:commons-lang:jar:2.6:compile
[INFO] +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] +- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.10:provided
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- xmlenc:xmlenc:jar:0.52:compile
[INFO] +- io.netty:netty-all:jar:4.0.23.Final:compile
[INFO] +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] +- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
{noformat}
As I mentioned earlier I plan to keep the dependency of {{hadoop-common}} /
{{hadoop-auth}} for the first phase, which would allow us to get rid of the
following dependency in the client jar:
{noformat}
[INFO] +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | \-
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349483#comment-14349483
]
Haohui Mai commented on HDFS-6200:
--
Here is the proposal for the first step:
Summary: (1) the changes are backward compatible, and (2) the changes will be
done in an incremental way to minimize risks.
* Update the pom.xml to create a new module {{hadoop-hdfs-client}} and publish
it into maven repository
* The old {{hadoop-hdfs}} jar depends on the {{hadoop-hdfs-client}} jar. There
should be no changes for downstream applications.
* Move the client implementation from {{hadoop-hdfs}} to {{hadoop-hdfs-client}}
incrementally. This can be done in trunk and be reviewed.
* Once the move is finished, we announce that the applications can depend on
{{hadoop-hdfs-client}} only.
* In this jira we left {{hadoop-common}} untouched. We'll take care of it in a
separate jira.
Thoughts?
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349520#comment-14349520 ] Sean Busbey commented on HDFS-6200: --- Since this new artifact is opt-in (since clients would have to change to it), we could instead use it to build an aggregate jar with no transitive dependencies. For this approach, we should not make the old hadoop-hdfs depend on it (since there will presumably be some shaded or otherwise isolated version of third party libraries present). We could still do the move incrementally by relying on maven to build the artifact with just those classes we need from hadoop-hdfs. That way, extant downstream applications who want to keep the current behavior can keep depending on hadoop-hdfs (or hadoop-client or whatever), and downstream applications who want the improved client dependency can change. When we're ready for a breaking change, we similarly announce that downstream applications should not be relying on hadoop-hdfs. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349557#comment-14349557
]
Haohui Mai commented on HDFS-6200:
--
bq. we could instead use it to build an aggregate jar with no transitive
dependencies
bq. (since there will presumably be some shaded or otherwise isolated version
of third party libraries present).
This is orthogonal. I don't thing we need to mix these issues in this jira.
bq. we should not make the old hadoop-hdfs depend on it (since there will
presumably be some shaded or otherwise isolated version of third party
libraries present).
The dependency is a real one -- the webhdfs server on DN uses {{DFSClient}} to
read data from HDFS.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345554#comment-14345554 ] Vinod Kumar Vavilapalli commented on HDFS-6200: --- +1000 for this proposal! (Not looked at the patch) Reproducing my comments at HADOOP-11656. - Having a separate hdfs client JAR would vastly reduce the amount of classpath conflicts. We have seen that in practice when we moved from Hadoop-1 MR to YARN having a leaner client JAR avoided a whole lot of problems we had before even if wasn't perfectly done. - A lean client JAR is also a major help in how we rationalize stack wide rolling upgrades - today NameNode is on the classpath of ResourceManager and RegionServer even if it doesn't get used and so it is very hard to layout and upgrade bits easily. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345657#comment-14345657 ] Sanjay Radia commented on HDFS-6200: +++1 for this proposal. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342659#comment-14342659
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651333/HDFS-6200.007.patch
against trunk revision e9ac88a.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9692//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036281#comment-14036281 ] Haohui Mai commented on HDFS-6200: -- Rebased on to the latest trunk. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036611#comment-14036611
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651236/HDFS-6200.006.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal:
org.apache.hadoop.hdfs.web.TestWebHDFSAcl
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7164//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7164//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036700#comment-14036700
]
Steve Loughran commented on HDFS-6200:
--
# DFSConfigKeys should be public and contain those keys clients are expected to
use
# this is a good time to switch to SLF4J for the logging here, and drop
commons-logging
# HdfsFileStatus gets its public final declarations in the wrong order ...
again, this is a good time to fix it.
# JsonUtilClient uses {{org.mortbay.util.ajax.JSON}} to parse the json. Thus
should be replaced by Jackson, so we don't need the mortbay libs on the
classpath?
# {{WebHdfsFileSystem}} also uses mortbay JSON for parsing
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036751#comment-14036751
]
Haohui Mai commented on HDFS-6200:
--
{quote}
DFSConfigKeys should be public and contain those keys clients are expected to
use
this is a good time to switch to SLF4J for the logging here, and drop
commons-logging
HdfsFileStatus gets its public final declarations in the wrong order ...
again, this is a good time to fix it.
JsonUtilClient uses org.mortbay.util.ajax.JSON to parse the json. Thus should
be replaced by Jackson, so we don't need the mortbay libs on the classpath?
WebHdfsFileSystem also uses mortbay JSON for parsing
{quote}
It might be better to restrict this patch to moving the files only. I plan to
address them in separate jiras. Filed HDFS-6564, HDFS-6565, HDFS-6566, and
HDFS-6567 to track them.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036888#comment-14036888
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12651333/HDFS-6200.007.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7174//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962655#comment-13962655
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639135/HDFS-6200.004.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6615//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6615//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6615//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962692#comment-13962692 ] Haohui Mai commented on HDFS-6200: -- The v5 patch fixes the audit warning. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962802#comment-13962802
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639158/HDFS-6200.005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6618//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6618//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch,
HDFS-6200.005.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963345#comment-13963345 ] Haohui Mai commented on HDFS-6200: -- [~szetszwo], [~tucu00], and [~ste...@apache.org], can you please take a look at this patch? Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch, HDFS-6200.005.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962402#comment-13962402
]
Haohui Mai commented on HDFS-6200:
--
The v0 patch moves WebHdfsFileSystem and SWebHdfsFileSystem into a separate
jar. The patch moves the files to a different project except for the following:
# It modifies {{DFSConfigKeys}} so that it no longer depends on the
{{AuthFilter}} and the {{BlockPlacementPolicyDefault}} class.
# It moves some methods from the {{HAUtil}} / {{DFSUtilClient}} in the original
hdfs jar to the {{HAUtilClient}} / {{DFSUtilClient}} in the client side jar.
# It moves some methods from the {{JsonUtils}} in the original hdfs jar to the
{{JsonUtilClient}} in the client side jar. It contains a new function that
translates JSON representation of the {{BlockLocation}} object to the Java
object.
To review this patch, one can use {{git diff -M}} to omit the renames.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962406#comment-13962406
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639093/HDFS-6200.000.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6610//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962443#comment-13962443
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639097/HDFS-6200.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6611//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962552#comment-13962552
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639104/HDFS-6200.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 1.3.9) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client
hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal:
org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6612//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6612//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962562#comment-13962562 ] Haohui Mai commented on HDFS-6200: -- The v3 patch fixes the findbugs and the unit test issues. Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962565#comment-13962565
]
Hadoop QA commented on HDFS-6200:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639133/HDFS-6200.003.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6614//console
This message is automatically generated.
Create a separate jar for hdfs-client
-
Key: HDFS-6200
URL: https://issues.apache.org/jira/browse/HDFS-6200
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch,
HDFS-6200.002.patch, HDFS-6200.003.patch
Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs
client. As discussed in the hdfs-dev mailing list
(http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to
access hdfs. The additional dependency sometimes can be difficult to manage
for projects like Apache Falcon and Apache Oozie.
This jira proposes to create a new project, hadoop-hdfs-cliient, which
contains the client side of the hdfs code. Downstream projects can use this
jar instead of the hadoop-hdfs to avoid unnecessary dependency.
Note that it does not break the compatibility of downstream projects. This is
because old downstream projects implicitly depend on hadoop-hdfs-client
through the hadoop-hdfs jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962569#comment-13962569 ] Haohui Mai commented on HDFS-6200: -- Rebased Create a separate jar for hdfs-client - Key: HDFS-6200 URL: https://issues.apache.org/jira/browse/HDFS-6200 Project: Hadoop HDFS Issue Type: Improvement Reporter: Haohui Mai Assignee: Haohui Mai Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch, HDFS-6200.004.patch Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie. This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency. Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar. -- This message was sent by Atlassian JIRA (v6.2#6252)
