[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-1.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Attachments: OOZIE-2512-1.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-2.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-3.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-1.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-1.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: (was: OOZIE-2512-amend-1.patch) > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-2.patch Amend patch to correct the test case and making sure that properties such as in hive-site.xml as getting cached. > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-3.patch The test failed because there is no symlink in hadoop-1. Skipping the assert in case of hadoop-1 > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-4.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-5.patch Amend-5: We have two maps: {code} shareLibConfigMap> shareLibMap> {code} The paths present in List are used to find configuration from {{shareLibConfigMap}}. It is obvious that values of Path should be the same in both to get the intended Configuration. Currently, its not the case. if in metafile we have {code} oozie.myhive=hdfs:///user/test/hive-site.xml#hive-site.xml {code} We populate map something like this - {code} shareLibConfigMap<"myhive", > shareLibMap < "myhive", [hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml] {code} In JavaActionExecutor, we need to convert {{hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml }}to {{ /user/test/hive-site.xml }}for getting the correct configuration from {{shareLibConfigMap}} Here is the code for it https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L682-L685 It works because {{hdfs://localhost:8020/user/test/hive-site.xml#hive-site.xml}} contains the fragment and we go to else part of the condition creating URI and getting Path part of it which is {{/user/test/hive-site.xml}} {code} new Path(new URI( actionLibPath.toString()).getPath()); {code} But in case if metafile is {code} oozie.myhive=/user/test/hive-site.xml {code} Then our maps : {code} shareLibConfigMap<"myhive", > shareLibMap < "myhive", [hdfs://localhost:8020/user/test/hive-site.xml] {code} In JavaActionExecutor, we won't go to the else part in the condition I showed. It simply refers to the {{hdfs://localhost:8020/user/test/hive-site.xml}} and does not get corresponding configuration in {{shareLibConfigMap}} Earlier it used to work because, we were not putting hdfs paths in {{shareLibMap}}, we simply put the path mentioned in metafile. I think we should be consistent with both the maps i.e. the Path we are putting in both should be the same and should not require any kind of logic for converting one Path for looking up into the other map to get corresponding Configuration. > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: trunk > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2512: -- Fix Version/s: (was: trunk) 4.3.0 > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 4.3.0 > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: OOZIE-2512-amend-5.patch > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 4.3.0 > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2512) ShareLibservice returns incorrect path for jar
[ https://issues.apache.org/jira/browse/OOZIE-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Subhashrao Saley updated OOZIE-2512: --- Attachment: (was: OOZIE-2512-amend-5.patch) > ShareLibservice returns incorrect path for jar > -- > > Key: OOZIE-2512 > URL: https://issues.apache.org/jira/browse/OOZIE-2512 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 4.3.0 > > Attachments: OOZIE-2512-1.patch, OOZIE-2512-2.patch, > OOZIE-2512-3.patch, OOZIE-2512-amend-1.patch, OOZIE-2512-amend-2.patch, > OOZIE-2512-amend-3.patch, OOZIE-2512-amend-4.patch, OOZIE-2512-amend-5.patch > > > If we have {{oozie.service.ShareLibService.mapping.file}} setting pointing to > a metafile, then ShareLibServe loads paths from that file. > We can mention path to a directory or path to a direct file. > When path to a directory is mentioned, the paths are populated correctly, but > not when we mentioned direct path for a file. > Consider following paths: > * /sharelib/pig/ > ** pig.jar > ** some.jar > * /sharelib/spark > ** spark-assembly.jar > In metafile, we have > {code} > oozie.pig=/sharelib/pig/ > oozie.spark=/sharelib/spark/spark-assembly.jar > {code} > Now, the SharelibService calculates the paths as > pig - > hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar > spark - /sharelib/spark/spark-assembly.jar > The spark path does not have hdfs prefixed. > Later on, when we run a spark action, it fails with > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/sharelib/spark/spark-assembly.jar does not exist > java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar > does not exist > {code} > Remember, if we already mentioned hdfs prefixed paths in metafile, then it > works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)