[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13302: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[
https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elek, Marton updated HDFS-13302:
Status: Patch Available (was: Open)
The easy workaround is implemented with moving the copy to a later position.
But long term a more mature solution may be required. For example if any of the
tools would use a dependency which is already used by the hdsl it won't be
included in the tools directory.
As of now it works well. I tested with both ozone/hdsl and yarn based tests:
{code}
==
Yarn
==
Yarn.Yarn Smoketest :: Smoketesting Yarn cluster
==
Daemons are running without error | PASS |
--
Calculate PI with mapreduce job | PASS |
--
Calculate word counts | PASS |
--
Yarn.Yarn Smoketest :: Smoketesting Yarn cluster | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==
{code}
{code}
==
Robot.Simple Cluster Topology :: Smoke test to start cluster with docker-co...
==
Daemons are running without error | PASS |
--
Check if datanode is connected to the scm | PASS |
--
Scale it up to 5 datanodes| PASS |
--
Test rest interface | PASS |
--
Check webui static resources | PASS |
--
Start freon testing | PASS |
--
Robot.Simple Cluster Topology :: Smoke test to start cluster with ... | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
{code}
> Ozone: fix classpath of yarn components
> ---
>
> Key: HDFS-13302
> URL: https://issues.apache.org/jira/browse/HDFS-13302
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: HDFS-7240
>Affects Versions: HDFS-7240
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDFS-13302-HDFS-7240.001.patch
>
>
> HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components.
> The reason is the behaviour of the dist-layout-stitching. The internal copy
> command in the dist-layout-stitching copies the jar files only if they don
> not exist in any of the existing subfolder of the share/hadoop directory.
> With the new separated classpath if some of the dependencies are already
> copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to
> the share/hadoop/yarn directory and won't be added to the classpath. (as the
> hdsl copy is before the yarn copy commands)
> Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the
> end of the dist-layout-stitching.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-13302: Attachment: HDFS-13302-HDFS-7240.001.patch > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
