[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778227#comment-16778227 ] Hadoop QA commented on OOZIE-3440: -- Testing JIRA OOZIE-3440 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any star imports .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} Javadoc generation succeeded with the patch .{color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) .{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. .{color:green}+1{color} There are no new bugs found in [sharelib/hive2]. .{color:green}+1{color} There are no new bugs found in [sharelib/spark]. .{color:green}+1{color} There are no new bugs found in [sharelib/oozie]. .{color:green}+1{color} There are no new bugs found in [sharelib/pig]. .{color:green}+1{color} There are no new bugs found in [sharelib/streaming]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive]. .{color:green}+1{color} There are no new bugs found in [sharelib/distcp]. .{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. .{color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. .{color:green}+1{color} There are no new bugs found in [sharelib/git]. .{color:green}+1{color} There are no new bugs found in [client]. .{color:green}+1{color} There are no new bugs found in [docs]. .{color:green}+1{color} There are no new bugs found in [tools]. .{color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. .{color:green}+1{color} There are no new bugs found in [server]. .{color:green}+1{color} There are no new bugs found in [webapp]. .{color:green}+1{color} There are no new bugs found in [examples]. .{color:green}+1{color} There are no new bugs found in [core]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 3140 .{color:orange}Tests failed at first run:{color} TestPurgeXCommand#testSucceededWorkflowSucceededSubWorkflow .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1023/ > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch, > OOZIE-3440-003.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a >
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778089#comment-16778089 ] Hadoop QA commented on OOZIE-3440: -- PreCommit-OOZIE-Build started > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch, > OOZIE-3440-003.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777823#comment-16777823 ] Andras Salamon commented on OOZIE-3440: --- Thanks [~dionusos] for the new patch, I left a few comments on the review board. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777664#comment-16777664 ] Denes Bodo commented on OOZIE-3440: --- Thanks [~asalamon74] for your comments, and thanks [~nobigo] for the testing. I've made the requested changes, please see the review board. Thanks. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776950#comment-16776950 ] Hadoop QA commented on OOZIE-3440: -- Testing JIRA OOZIE-3440 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any star imports .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} Javadoc generation succeeded with the patch .{color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) .{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. .{color:green}+1{color} There are no new bugs found in [sharelib/hive2]. .{color:green}+1{color} There are no new bugs found in [sharelib/spark]. .{color:green}+1{color} There are no new bugs found in [sharelib/oozie]. .{color:green}+1{color} There are no new bugs found in [sharelib/pig]. .{color:green}+1{color} There are no new bugs found in [sharelib/streaming]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive]. .{color:green}+1{color} There are no new bugs found in [sharelib/distcp]. .{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. .{color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. .{color:green}+1{color} There are no new bugs found in [sharelib/git]. .{color:green}+1{color} There are no new bugs found in [client]. .{color:green}+1{color} There are no new bugs found in [docs]. .{color:green}+1{color} There are no new bugs found in [tools]. .{color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. .{color:green}+1{color} There are no new bugs found in [server]. .{color:green}+1{color} There are no new bugs found in [webapp]. .{color:green}+1{color} There are no new bugs found in [examples]. .{color:green}+1{color} There are no new bugs found in [core]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 3140 .{color:orange}Tests failed at first run:{color} TestPurgeXCommand#testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1019/ > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a >
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776844#comment-16776844 ] Hadoop QA commented on OOZIE-3440: -- PreCommit-OOZIE-Build started > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch, OOZIE-3440-002.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776707#comment-16776707 ] Andras Salamon commented on OOZIE-3440: --- [~nobigo] Thanks for the test [~dionusos] I left a few comments on review board. Mostly test enhancements. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776479#comment-16776479 ] duan xiong commented on OOZIE-3440: --- HI,[~asalamon74],I have test the patch uploaded by [~dionusos]. It is still works after the modification. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776464#comment-16776464 ] duan xiong commented on OOZIE-3440: --- [~asalamon74],Ok, I will try to patch it and do it again. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775302#comment-16775302 ] Hadoop QA commented on OOZIE-3440: -- Testing JIRA OOZIE-3440 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any star imports .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} Javadoc generation succeeded with the patch .{color:green}+1{color} the patch does not seem to introduce new Javadoc warning(s) .{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s) {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. .{color:green}+1{color} There are no new bugs found in [fluent-job/fluent-job-api]. .{color:green}+1{color} There are no new bugs found in [docs]. .{color:green}+1{color} There are no new bugs found in [examples]. .{color:green}+1{color} There are no new bugs found in [webapp]. .{color:green}+1{color} There are no new bugs found in [tools]. .{color:green}+1{color} There are no new bugs found in [client]. .{color:green}+1{color} There are no new bugs found in [server]. .{color:green}+1{color} There are no new bugs found in [core]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive2]. .{color:green}+1{color} There are no new bugs found in [sharelib/streaming]. .{color:green}+1{color} There are no new bugs found in [sharelib/pig]. .{color:green}+1{color} There are no new bugs found in [sharelib/hive]. .{color:green}+1{color} There are no new bugs found in [sharelib/oozie]. .{color:green}+1{color} There are no new bugs found in [sharelib/spark]. .{color:green}+1{color} There are no new bugs found in [sharelib/git]. .{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. .{color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. .{color:green}+1{color} There are no new bugs found in [sharelib/distcp]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 3137 {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/1018/ > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775154#comment-16775154 ] Denes Bodo commented on OOZIE-3440: --- I've updated the environment info: HDP 2.6.5, Hadoop 2.7.3 and Spark 2 . 2.3.0 . Oozie parameters the Spark command's --files option with the "%23" contained string. This can be reproduces using unit tests. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775160#comment-16775160 ] Hadoop QA commented on OOZIE-3440: -- PreCommit-OOZIE-Build started > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775157#comment-16775157 ] Andras Salamon commented on OOZIE-3440: --- [~nobigo] Can you please test the patch uploaded by [~dionusos]. Is your {{hdfs:///var/log/nobigo/property.yml#property.yml}} example still works after the modification? > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 2.7.3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775150#comment-16775150 ] duan xiong commented on OOZIE-3440: --- Hi,[~dionusos],I use hadoop version 2.6.5 and Spark version . 2.1.0. May be because version different. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 > Environment: Hadoop 3 > Spark 2 - 2.3.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775146#comment-16775146 ] Denes Bodo commented on OOZIE-3440: --- [~nobigo] have you used spark 1 or spark 2? Can you please share the Hadoop version too? Thanks > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > Attachments: OOZIE-3440-001.patch > > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775118#comment-16775118 ] duan xiong commented on OOZIE-3440: --- Hi,[~dionusos], I tried add file tag right address "hdfs:///var/log/nobigo/property.yml#property.yml" in my workflow file. It succeed. I add wrong address "hdfs:///var/log/nobigo/property.yml1#property.yml".It failed. It shows: JA008:file doies not exist. I use new Oozie code compile oozie-5.2.0-SNAPSHOT. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775074#comment-16775074 ] duan xiong commented on OOZIE-3440: --- Hi,[~dionusos], I tried add file tag wright address "hdfs:///var/log/nobigo/log/application0004#application0004" in my workflow file. It succeed. I add wrong address "hdfs:///var/log/nobigo/log/application0003#application0003".It failed. It shows: JA008:file doies not exist. I use new Oozie code compile oozie-5.2.0-SNAPSHOT. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3440) Oozie Spark action replaces path symlink # to %23
[ https://issues.apache.org/jira/browse/OOZIE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774908#comment-16774908 ] Denes Bodo commented on OOZIE-3440: --- Hallo [~nobigo], thanks for your answer. We tried hdfs://, I've just extended the description. This fails too. > Oozie Spark action replaces path symlink # to %23 > - > > Key: OOZIE-3440 > URL: https://issues.apache.org/jira/browse/OOZIE-3440 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 4.3.1, 5.1.0 >Reporter: Denes Bodo >Assignee: Denes Bodo >Priority: Major > > When we provide in a hive action: > * --files /etc/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml or > * hdfs:///tmp/spark2/conf/hive-site.xml#hive-site.xml or > * /etc/spark2/conf/hive-site.xml#hive-site.xml > we get the following error: > {code} > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File > file:/etc/spark2/conf/hive-site.xml%23hive-site.xml does not exist > {code} > The culprit seems to be > https://github.com/apache/oozie/blob/master/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkArgsExtractor.java#L480L489 > . > Please help me confirm if this is a bug or not. Meanwhile I am creating a > fix/workaround to this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)