[
https://issues.apache.org/jira/browse/TEZ-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072536#comment-14072536
]
Siddharth Seth commented on TEZ-1300:
-------------------------------------
Thanks for the review.
bq. website should have the install page updated
bq. please rename the current install page to a 0.4 specific install page for
users downloading the 0.4.x release
will do.
bq. tez.use.cluster.hadoop-libs - this should not be needed to be set for the
recommended install approach.
I'll change the documentation to say - this should not be set / set to false.
bq. Not sure as to why the partial approach does not use a tarball?
This is something which needs to be figured out. Currently we have three
targets. 1) full - which depending on command line options generates either a
dir / tar.gz etc. 2) partial - which also depends on the same command line
option. 3) full-tarball which always generates a .tgz file.
I think the third option is a must have. A partial build is also useful. Do you
think we should remove the option in 1/2 to create a dir / .tgz and instead
always generate a tarball (The first one can be removed in that case).
bq. After this patch, why do we still need to support a partial install? Also,
does the partial install continue to have the mapreduce jars that are needed
directly by Tez?
That's to support users who still want to use hadoop-libraries from the
cluster. The mapreduce jars are included - since tez directly depends on them.
bq. how will this work for a full tarball as this will create a mix of hadoop
jars ( some from the hadoop install and some from the tez install)
Do you have a suggestion on fixing this ? We use HADOOP_CLASSPATH_FIRST, which
should work - even if there's two copies of jars. One possible option is to
specify the partial build as the one which needs to be setup on the client.
bq. is this to be addressed in a follow up jira?
Will remove the comment. Already verified the behaviour.
bq. should this leverage the default fs if not absolute?
Follow up jira, if we want to change this behaviour. As of now, we expect a
fully qualified name.
bq. public static String getFrameworkClasspath - shouldn't the tez jars be at
the front of the classpath i.e before PWD.
Explicitly put it at the end - so that user libraries are in the classpath
before tez libs.
bq. TezConfiguration.TEZ_USE_CLUSTER_HADOOP_LIBS - does this make sense if the
full tarball is used?
Required - since a partial tez-build is still supported.
bq. javadocs in TezConf mention tez but code seems to be using tezlib
Some more details please ?
> Change default tez classpath to not include hadoop jars from the cluster
> ------------------------------------------------------------------------
>
> Key: TEZ-1300
> URL: https://issues.apache.org/jira/browse/TEZ-1300
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-1300.1.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
