[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597121#comment-16597121 ] shivusondur commented on SPARK-25271: - !image-2018-08-30-12-26-07-502.png! > Creati

[jira] [Issue Comment Deleted] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shivusondur updated SPARK-25271: Comment: was deleted (was: !image-2018-08-30-12-26-07-502.png!) > Creating parquet table with all

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2018-08-29 Thread Darcy Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597082#comment-16597082 ] Darcy Shen commented on SPARK-14220: Jenkins for Scala 2.12:   https://amplab.cs.b

[jira] [Assigned] (SPARK-25276) Redundant constrains when using alias

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25276: Assignee: Apache Spark > Redundant constrains when using alias >

[jira] [Commented] (SPARK-25276) Redundant constrains when using alias

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597027#comment-16597027 ] Apache Spark commented on SPARK-25276: -- User 'ajithme' has created a pull request f

[jira] [Assigned] (SPARK-25276) Redundant constrains when using alias

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25276: Assignee: (was: Apache Spark) > Redundant constrains when using alias > -

[jira] [Updated] (SPARK-25276) Redundant constrains when using alias

2018-08-29 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-25276: Attachment: test.patch > Redundant constrains when using alias > - > >

[jira] [Created] (SPARK-25276) Redundant constrains when using alias

2018-08-29 Thread Ajith S (JIRA)
Ajith S created SPARK-25276: --- Summary: Redundant constrains when using alias Key: SPARK-25276 URL: https://issues.apache.org/jira/browse/SPARK-25276 Project: Spark Issue Type: Bug Compone

[jira] [Updated] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

2018-08-29 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24785: Fix Version/s: 2.4.0 > Making sure REPL prints Spark UI info and then Welcome message > --

[jira] [Created] (SPARK-25275) require memberhip in wheel to run 'su' (in dockerfiles)

2018-08-29 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-25275: -- Summary: require memberhip in wheel to run 'su' (in dockerfiles) Key: SPARK-25275 URL: https://issues.apache.org/jira/browse/SPARK-25275 Project: Spark I

[jira] [Commented] (SPARK-25217) Error thrown when creating BlockMatrix

2018-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596992#comment-16596992 ] Liang-Chi Hsieh commented on SPARK-25217: - If no further question, I think we ca

[jira] [Comment Edited] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596949#comment-16596949 ] Liang-Chi Hsieh edited comment on SPARK-25271 at 8/30/18 12:11 AM: ---

[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596949#comment-16596949 ] Liang-Chi Hsieh commented on SPARK-25271: - I think this is a known issue on Hive

[jira] [Resolved] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

2018-08-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24909. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21976 [https:

[jira] [Assigned] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

2018-08-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24909: -- Assignee: Thomas Graves > Spark scheduler can hang when fetch failures, executor lost

[jira] [Commented] (SPARK-25242) Suggestion to make sql config setting fluent

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596930#comment-16596930 ] Apache Spark commented on SPARK-25242: -- User 'bomeng' has created a pull request fo

[jira] [Assigned] (SPARK-25242) Suggestion to make sql config setting fluent

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25242: Assignee: (was: Apache Spark) > Suggestion to make sql config setting fluent > --

[jira] [Assigned] (SPARK-25242) Suggestion to make sql config setting fluent

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25242: Assignee: Apache Spark > Suggestion to make sql config setting fluent > -

[jira] [Updated] (SPARK-25274) Improve toPandas with Arrow by sending out-of-order record batches

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-25274: - Summary: Improve toPandas with Arrow by sending out-of-order record batches (was: Improve `toPa

[jira] [Assigned] (SPARK-25274) Improve `toPandas` with Arrow by sending out-of-order record batches

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25274: Assignee: (was: Apache Spark) > Improve `toPandas` with Arrow by sending out-of-order

[jira] [Assigned] (SPARK-25274) Improve `toPandas` with Arrow by sending out-of-order record batches

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25274: Assignee: Apache Spark > Improve `toPandas` with Arrow by sending out-of-order record bat

[jira] [Commented] (SPARK-25274) Improve `toPandas` with Arrow by sending out-of-order record batches

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596891#comment-16596891 ] Apache Spark commented on SPARK-25274: -- User 'BryanCutler' has created a pull reque

[jira] [Commented] (SPARK-25167) Minor fixes for R sql tests (tests that fail in development environment)

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596864#comment-16596864 ] Apache Spark commented on SPARK-25167: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-25274) Improve `toPandas` with Arrow by sending out-of-order record batches

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596860#comment-16596860 ] Bryan Cutler commented on SPARK-25274: -- This is a followup to SPARK-23030 that is n

[jira] [Issue Comment Deleted] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-25272: - Comment: was deleted (was: This is a followup that is possible now that Arrow streaming format i

[jira] [Commented] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596859#comment-16596859 ] Bryan Cutler commented on SPARK-25272: -- This is a followup that is possible now tha

[jira] [Created] (SPARK-25274) Improve `toPandas` with Arrow by sending out-of-order record batches

2018-08-29 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-25274: Summary: Improve `toPandas` with Arrow by sending out-of-order record batches Key: SPARK-25274 URL: https://issues.apache.org/jira/browse/SPARK-25274 Project: Spark

[jira] [Updated] (SPARK-24005) Remove usage of Scala’s parallel collection

2018-08-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24005: Labels: (was: starter) > Remove usage of Scala’s parallel collection > -

[jira] [Updated] (SPARK-25273) How to install testthat v1.0.2

2018-08-29 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-25273: --- Summary: How to install testthat v1.0.2 (was: How to install testthat = 1.0.2) > How to install te

[jira] [Commented] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596850#comment-16596850 ] Bryan Cutler commented on SPARK-25272: -- yeah that would be great to make it easier

[jira] [Commented] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596844#comment-16596844 ] Apache Spark commented on SPARK-25272: -- User 'BryanCutler' has created a pull reque

[jira] [Assigned] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25272: Assignee: Apache Spark (was: Bryan Cutler) > Show some kind of test output to indicate p

[jira] [Assigned] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25272: Assignee: Bryan Cutler (was: Apache Spark) > Show some kind of test output to indicate p

[jira] [Assigned] (SPARK-25273) How to install testthat = 1.0.2

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25273: Assignee: (was: Apache Spark) > How to install testthat = 1.0.2 > ---

[jira] [Commented] (SPARK-25273) How to install testthat = 1.0.2

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596842#comment-16596842 ] Apache Spark commented on SPARK-25273: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-25273) How to install testthat = 1.0.2

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25273: Assignee: Apache Spark > How to install testthat = 1.0.2 > --

[jira] [Created] (SPARK-25273) How to install testthat = 1.0.2

2018-08-29 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-25273: -- Summary: How to install testthat = 1.0.2 Key: SPARK-25273 URL: https://issues.apache.org/jira/browse/SPARK-25273 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596836#comment-16596836 ] Imran Rashid commented on SPARK-25272: -- it would also be great if we could get the

[jira] [Updated] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-25272: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-22216 > Show some kind of test ou

[jira] [Created] (SPARK-25272) Show some kind of test output to indicate pyarrow tests were run

2018-08-29 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-25272: Summary: Show some kind of test output to indicate pyarrow tests were run Key: SPARK-25272 URL: https://issues.apache.org/jira/browse/SPARK-25272 Project: Spark

[jira] [Assigned] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25268: Assignee: Apache Spark > runParallelPersonalizedPageRank throws serialization Exception >

[jira] [Commented] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596676#comment-16596676 ] Apache Spark commented on SPARK-25268: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25268: Assignee: (was: Apache Spark) > runParallelPersonalizedPageRank throws serialization

[jira] [Commented] (SPARK-25267) Disable ConvertToLocalRelation in the test cases of sql/core and sql/hive

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596672#comment-16596672 ] Apache Spark commented on SPARK-25267: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-25267) Disable ConvertToLocalRelation in the test cases of sql/core and sql/hive

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25267: Assignee: Apache Spark > Disable ConvertToLocalRelation in the test cases of sql/core and

[jira] [Assigned] (SPARK-25267) Disable ConvertToLocalRelation in the test cases of sql/core and sql/hive

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25267: Assignee: (was: Apache Spark) > Disable ConvertToLocalRelation in the test cases of s

[jira] [Commented] (SPARK-25180) Spark standalone failure in Utils.doFetchFile() if nslookup of local hostname fails

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596566#comment-16596566 ] Steve Loughran commented on SPARK-25180: Reviewing a bit more, I think the root

[jira] [Issue Comment Deleted] (SPARK-25246) When the spark.eventLog.compress is enabled, the Application is not showing in the History server UI ('incomplete application' page), initially.

2018-08-29 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25246: --- Comment: was deleted (was: I am working on it :)) > When the spark.eventLog.compress is enabled, the Applic

[jira] [Commented] (SPARK-23997) Configurable max number of buckets

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596525#comment-16596525 ] Apache Spark commented on SPARK-23997: -- User 'kiszk' has created a pull request for

[jira] [Issue Comment Deleted] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-08-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25269: Comment: was deleted (was: I'm working on.) > SQL interface support specify StorageLevel when cac

[jira] [Commented] (SPARK-25253) Refactor pyspark connection & authentication

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596409#comment-16596409 ] Apache Spark commented on SPARK-25253: -- User 'cclauss' has created a pull request f

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596404#comment-16596404 ] Steve Loughran commented on SPARK-6305: --- bq. Could be possible that nobody is swapp

[jira] [Resolved] (SPARK-25266) Fix memory leak in Barrier Execution Mode

2018-08-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25266. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22258 [https://

[jira] [Commented] (SPARK-23298) distinct.count on Dataset/DataFrame yields non-deterministic results

2018-08-29 Thread Mateusz Jukiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596366#comment-16596366 ] Mateusz Jukiewicz commented on SPARK-23298: --- [~jansi] I can see two issues in

[jira] [Comment Edited] (SPARK-23298) distinct.count on Dataset/DataFrame yields non-deterministic results

2018-08-29 Thread Janne Jaanila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596324#comment-16596324 ] Janne Jaanila edited comment on SPARK-23298 at 8/29/18 1:27 PM: --

[jira] [Comment Edited] (SPARK-23298) distinct.count on Dataset/DataFrame yields non-deterministic results

2018-08-29 Thread Janne Jaanila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596324#comment-16596324 ] Janne Jaanila edited comment on SPARK-23298 at 8/29/18 1:26 PM: --

[jira] [Commented] (SPARK-23298) distinct.count on Dataset/DataFrame yields non-deterministic results

2018-08-29 Thread Janne Jaanila (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596324#comment-16596324 ] Janne Jaanila commented on SPARK-23298: --- I have faced a similar problem with multi

[jira] [Commented] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596300#comment-16596300 ] shivusondur commented on SPARK-25271: - cc [~cloud_fan] > Creating parquet table wit

[jira] [Created] (SPARK-25271) Creating parquet table with all the column null throws exception

2018-08-29 Thread shivusondur (JIRA)
shivusondur created SPARK-25271: --- Summary: Creating parquet table with all the column null throws exception Key: SPARK-25271 URL: https://issues.apache.org/jira/browse/SPARK-25271 Project: Spark

[jira] [Commented] (SPARK-25217) Error thrown when creating BlockMatrix

2018-08-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596250#comment-16596250 ] Liang-Chi Hsieh commented on SPARK-25217: - I think you are mixing {{pyspark.ml.l

[jira] [Assigned] (SPARK-25270) lint-python: Add flake8 to find syntax errors and undefined names

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25270: Assignee: Apache Spark > lint-python: Add flake8 to find syntax errors and undefined name

[jira] [Assigned] (SPARK-25270) lint-python: Add flake8 to find syntax errors and undefined names

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25270: Assignee: (was: Apache Spark) > lint-python: Add flake8 to find syntax errors and und

[jira] [Commented] (SPARK-25270) lint-python: Add flake8 to find syntax errors and undefined names

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596234#comment-16596234 ] Apache Spark commented on SPARK-25270: -- User 'cclauss' has created a pull request f

[jira] [Created] (SPARK-25270) lint-python: Add flake8 to find syntax errors and undefined names

2018-08-29 Thread cclauss (JIRA)
cclauss created SPARK-25270: --- Summary: lint-python: Add flake8 to find syntax errors and undefined names Key: SPARK-25270 URL: https://issues.apache.org/jira/browse/SPARK-25270 Project: Spark Issu

[jira] [Commented] (SPARK-25265) Fix memory leak vulnerability in Barrier Execution Mode

2018-08-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596221#comment-16596221 ] Marco Gaido commented on SPARK-25265: - Isn't this a duplicate of the next one? > Fi

[jira] [Commented] (SPARK-24716) Refactor ParquetFilters

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596176#comment-16596176 ] Apache Spark commented on SPARK-24716: -- User 'wangyum' has created a pull request f

[jira] [Commented] (SPARK-25102) Write Spark version information to Parquet file footers

2018-08-29 Thread Nikita Poberezkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596110#comment-16596110 ] Nikita Poberezkin commented on SPARK-25102: --- Hello, [~zi] I've tried this appr

[jira] [Commented] (SPARK-25219) KMeans Clustering - Text Data - Results are incorrect

2018-08-29 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596109#comment-16596109 ] Marco Gaido commented on SPARK-25219: - Well, there are many differences between Spar

[jira] [Assigned] (SPARK-25256) Plan mismatch errors in Hive tests in 2.12

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25256: Assignee: Apache Spark > Plan mismatch errors in Hive tests in 2.12 > ---

[jira] [Commented] (SPARK-25256) Plan mismatch errors in Hive tests in 2.12

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596073#comment-16596073 ] Apache Spark commented on SPARK-25256: -- User 'sadhen' has created a pull request fo

[jira] [Assigned] (SPARK-25256) Plan mismatch errors in Hive tests in 2.12

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25256: Assignee: (was: Apache Spark) > Plan mismatch errors in Hive tests in 2.12 >

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596070#comment-16596070 ] Apache Spark commented on SPARK-25044: -- User 'sadhen' has created a pull request fo

[jira] [Resolved] (SPARK-25259) Left/Right join support push down during-join predicates

2018-08-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25259. - Resolution: Duplicate > Left/Right join support push down during-join predicates > -

[jira] [Commented] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

2018-08-29 Thread chen xiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596033#comment-16596033 ] chen xiao commented on SPARK-25193: --- It does look like the Hive issue. Thanks [~mgaido

[jira] [Resolved] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25044. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22259 [https://gith

[jira] [Assigned] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25044: --- Assignee: Sean Owen > Address translation of LMF closure primitive args to Object in Scala

[jira] [Resolved] (SPARK-23030) Decrease memory consumption with toPandas() collection using Arrow

2018-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23030. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21546 [https://gi

[jira] [Assigned] (SPARK-23030) Decrease memory consumption with toPandas() collection using Arrow

2018-08-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23030: Assignee: Bryan Cutler > Decrease memory consumption with toPandas() collection using Arr