[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-1.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2512-1.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-2.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-3.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-1.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-1.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: (was: OOZIE-2512-amend-1.patch)
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-2.patch
Amend patch to correct the test case and making sure that properties such as in
hive-site.xml as getting cached.
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-3.patch
The test failed because there is no symlink in hadoop-1. Skipping the assert in
case of hadoop-1
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-4.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-5.patch
Amend-5:
We have two maps:
{code}
shareLibConfigMap>
shareLibMap>
{code}
The paths present in List are used to find configuration from
{{shareLibConfigMap}}. It is obvious that values of Path should be the same in
both to get the intended Configuration.
Currently, its not the case.
if in metafile we have
{code}
oozie.myhive=hdfs:///user/test/hive-site.xml#hive-site.xml
{code}
We populate map something like this -
{code}
shareLibConfigMap<"myhive", >
shareLibMap < "myhive",
[hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml]
{code}
In JavaActionExecutor, we need to convert
{{hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml }}to {{
/user/test/hive-site.xml }}for getting the correct configuration from
{{shareLibConfigMap}}
Here is the code for it
https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L682-L685
It works because
{{hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml}} contains the
fragment and we go to else part of the condition creating URI and getting Path
part of it which is {{/user/test/hive-site.xml}}
{code}
new Path(new URI(
actionLibPath.toString()).getPath());
{code}
But in case if metafile is
{code}
oozie.myhive=/user/test/hive-site.xml
{code}
Then our maps :
{code}
shareLibConfigMap<"myhive", >
shareLibMap < "myhive", [hdfs://localhost:8020/user/test/hive-site.xml]
{code}
In JavaActionExecutor, we won't go to the else part in the condition I showed.
It simply refers to the {{hdfs://localhost:8020/user/test/hive-site.xml}} and
does not get corresponding configuration in {{shareLibConfigMap}}
Earlier it used to work because, we were not putting hdfs paths in
{{shareLibMap}}, we simply put the path mentioned in metafile.
I think we should be consistent with both the maps i.e. the Path we are putting
in both should be the same and should not require any kind of logic for
converting one Path for looking up into the other map to get corresponding
Configuration.
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: trunk
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2512:
--
Fix Version/s: (was: trunk)
4.3.0
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: OOZIE-2512-amend-5.patch
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[
https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2512:
---
Attachment: (was: OOZIE-2512-amend-5.patch)
> ShareLibservice returns incorrect path for jar
> --
>
> Key: OOZIE-2512
> URL: https://issues.apache.org/jira/browse/OOZIE-2512
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch,
> OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch,
> OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch
>
>
> If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to
> a metafile, then ShareLibServe loads paths from that file.
> We can mention path to a directory or path to a direct file.
> When path to a directory is mentioned, the paths are populated correctly, but
> not when we mentioned direct path for a file.
> Consider following paths:
> * /sharelib/pig/
> ** pig.jar
> ** some.jar
> * /sharelib/spark
> ** spark-assembly.jar
> In metafile, we have
> {code}
> oozie.pig=/sharelib/pig/
> oozie.spark=/sharelib/spark/spark-assembly.jar
> {code}
> Now, the SharelibService calculates the paths as
> pig -
> hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
> spark - /sharelib/spark/spark-assembly.jar
> The spark path does not have hdfs prefixed.
> Later on, when we run a spark action, it fails with
> {code}
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File
> file:/sharelib/spark/spark-assembly.jar does not exist
> java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar
> does not exist
> {code}
> Remember, if we already mentioned hdfs prefixed paths in metafile, then it
> works fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
