[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472681#comment-15472681 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464236#comment-15464236 ] Hyukjin Kwon commented on SPARK-17339: -- [~sarutak] [~shivaram] Please cc me if any of you submit a PR so that I can run the build automation (as it is not merged yet). Otherwise, I can do this if you tell me which one is preferred. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472682#comment-15472682 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464236#comment-15464236 ] Hyukjin Kwon commented on SPARK-17339: -- [~sarutak] [~shivaram] Please cc me if any of you submit a PR so that I can run the build automation (as it is not merged yet). Otherwise, I can do this if you tell me which one is preferred. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472604#comment-15472604 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464244#comment-15464244 ] Shivaram Venkataraman commented on SPARK-17339: --- Thanks [~hyukjin.kwon] -- It will be great if you can try the `Utils.resolveURI` change as a PR and run that through the build automation tool. Also the reason I was trying to debug this today is I feel like it would be better to make the build green before merging the automation -- otherwise it might confuse other contributors etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472602#comment-15472602 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464244#comment-15464244 ] Shivaram Venkataraman commented on SPARK-17339: --- Thanks [~hyukjin.kwon] -- It will be great if you can try the `Utils.resolveURI` change as a PR and run that through the build automation tool. Also the reason I was trying to debug this today is I feel like it would be better to make the build green before merging the automation -- otherwise it might confuse other contributors etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472549#comment-15472549 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464247#comment-15464247 ] Kousuke Saruta commented on SPARK-17339: [~hyukjin.kwon] Go ahead and submit a PR. Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472551#comment-15472551 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464247#comment-15464247 ] Kousuke Saruta commented on SPARK-17339: [~hyukjin.kwon] Go ahead and submit a PR. Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472523#comment-15472523 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464250#comment-15464250 ] Hyukjin Kwon commented on SPARK-17339: -- Yeap, I totally agree. Thank you both! Will submit a PR within today. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472522#comment-15472522 ] Hadoop QA commented on SPARK-17339: --- [ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464250#comment-15464250 ] Hyukjin Kwon commented on SPARK-17339: -- Yeap, I totally agree. Thank you both! Will submit a PR within today. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472245#comment-15472245 ] Apache Spark commented on SPARK-17339: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/15008 > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman >Assignee: Hyukjin Kwon > Fix For: 2.1.0 > > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464815#comment-15464815 ] Apache Spark commented on SPARK-17339: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/14960 > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464250#comment-15464250 ] Hyukjin Kwon commented on SPARK-17339: -- Yeap, I totally agree. Thank you both! Will submit a PR within today. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464247#comment-15464247 ] Kousuke Saruta commented on SPARK-17339: [~hyukjin.kwon] Go ahead and submit a PR. Thanks! > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464244#comment-15464244 ] Shivaram Venkataraman commented on SPARK-17339: --- Thanks [~hyukjin.kwon] -- It will be great if you can try the `Utils.resolveURI` change as a PR and run that through the build automation tool. Also the reason I was trying to debug this today is I feel like it would be better to make the build green before merging the automation -- otherwise it might confuse other contributors etc. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464236#comment-15464236 ] Hyukjin Kwon commented on SPARK-17339: -- [~sarutak] [~shivaram] Please cc me if any of you submit a PR so that I can run the build automation (as it is not merged yet). Otherwise, I can do this if you tell me which one is preferred. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464193#comment-15464193 ] Kousuke Saruta commented on SPARK-17339: [~shivaram] I think more discussion is needed for the better solution of SPARK-11227. For hot-fix, what you mentioned is one solution and another can be using `org.apache.spark.util.Utils.resolveURI`. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464154#comment-15464154 ] Shivaram Venkataraman commented on SPARK-17339: --- Specifically can we construct a Hadoop `Path` and then get the URI from that Path ? The reason I think this might work for the SparkR case is that textFile is able to read that file after it is converted to a Path > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[
https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464144#comment-15464144
]
Shivaram Venkataraman commented on SPARK-17339:
---
[~felixcheung] [~hyukjin.kwon] I looked into this a little bit more today and I
think the failures are related to SPARK-11227 (or rather the PR at
https://github.com/apache/spark/pull/13738/files) -- Specifically the line
which calls `FileSystem.get(new URI(path), hadoopConfiguration)` in
`hadoopFile` leads to failures with a stack trace that looks like.
{code}
java.net.URISyntaxException: Illegal character in opaque part at index 2:
C:\Users\shivaram\Downloads\spark-1-sparkr-win-tests-fix\spark-1-sparkr-win-tests-fix\README.md
at java.net.URI$Parser.fail(URI.java:2829)
at java.net.URI$Parser.checkChars(URI.java:3002)
at java.net.URI$Parser.parse(URI.java:3039)
at java.net.URI.(URI.java:595)
at
org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:995)
at
org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:990)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:686)
at org.apache.spark.SparkContext.hadoopFile(SparkContext.scala:990)
at
org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:806)
{code}
If I comment out the changes from that PR, the SparkR tests seem to pass.
[~sarutak] Do you know if there is a better way to fix the issue in SPARK-11227
without trying to construct a URI from the path ?
> Fix SparkR tests on Windows
> ---
>
> Key: SPARK-17339
> URL: https://issues.apache.org/jira/browse/SPARK-17339
> Project: Spark
> Issue Type: Bug
> Components: SparkR, Tests
>Reporter: Shivaram Venkataraman
>
> A number of SparkR tests are current failing when run on Windows as discussed
> in https://github.com/apache/spark/pull/14743
> The list of tests that fail right now is at
> https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134
> A full log from a build and test on AppVeyor is at
> https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[
https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453238#comment-15453238
]
Shivaram Venkataraman commented on SPARK-17339:
---
Looking at the code my current guess is that calling normalizePath might result
in file paths that dont work in Windows. The R function has an option
"winslash: the separator to be used on Windows - ignored elsewhere. Must be one
of ‘c("/", "\\")’." Right now we are using the default which is "\\"
> Fix SparkR tests on Windows
> ---
>
> Key: SPARK-17339
> URL: https://issues.apache.org/jira/browse/SPARK-17339
> Project: Spark
> Issue Type: Bug
> Components: SparkR, Tests
>Reporter: Shivaram Venkataraman
>
> A number of SparkR tests are current failing when run on Windows as discussed
> in https://github.com/apache/spark/pull/14743
> The list of tests that fail right now is at
> https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134
> A full log from a build and test on AppVeyor is at
> https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453139#comment-15453139 ] Shivaram Venkataraman commented on SPARK-17339: --- We are discussing automating Windows builds in https://github.com/apache/spark/pull/14859 -- so we'll have some way to test this if we have a proposed fix. But yeah it might still be hard to debug / trace why this is happening > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452998#comment-15452998 ] Felix Cheung commented on SPARK-17339: -- Seems that way. I can help but I don't have Windows to test this.. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows
[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452671#comment-15452671 ] Shivaram Venkataraman commented on SPARK-17339: --- >From a cursory look, most of the errors seem to be related to file paths when >loading / saving data. I think we can probably fix them in one go, but if we >want we can also split this into smaller sub-tasks. > Fix SparkR tests on Windows > --- > > Key: SPARK-17339 > URL: https://issues.apache.org/jira/browse/SPARK-17339 > Project: Spark > Issue Type: Bug > Components: SparkR, Tests >Reporter: Shivaram Venkataraman > > A number of SparkR tests are current failing when run on Windows as discussed > in https://github.com/apache/spark/pull/14743 > The list of tests that fail right now is at > https://gist.github.com/shivaram/7693df7bd54dc81e2e7d1ce296c41134 > A full log from a build and test on AppVeyor is at > https://ci.appveyor.com/project/HyukjinKwon/spark/build/46-test123 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
