[jira] [Resolved] (SPARK-25055) MessageWithHeader transfer ByteBuffer from Netty's CompositeByteBuf many times

2018-08-08 Thread DENG FEI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DENG FEI resolved SPARK-25055. -- Resolution: Duplicate > MessageWithHeader transfer ByteBuffer from Netty's CompositeByteBuf many times

[jira] [Resolved] (SPARK-25058) Use Block.isEmpty/nonEmpty to check whether the code is empty or not.

2018-08-08 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-25058. --- Resolution: Fixed Assignee: Takuya Ueshin Fix Version/s: 2.4.0 Issue resolve

[jira] [Commented] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

2018-08-08 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574276#comment-16574276 ] Steve Loughran commented on SPARK-23050: bq. Is there any way we can avoid happe

[jira] [Commented] (SPARK-25063) Rename class KnowNotNull to KnownNotNull

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574275#comment-16574275 ] Apache Spark commented on SPARK-25063: -- User 'maryannxue' has created a pull reques

[jira] [Assigned] (SPARK-25063) Rename class KnowNotNull to KnownNotNull

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25063: Assignee: Apache Spark > Rename class KnowNotNull to KnownNotNull > -

[jira] [Assigned] (SPARK-25063) Rename class KnowNotNull to KnownNotNull

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25063: Assignee: (was: Apache Spark) > Rename class KnowNotNull to KnownNotNull > --

[jira] [Updated] (SPARK-25063) Rename class KnowNotNull to KnownNotNull

2018-08-08 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-25063: Description: It's a class name typo checked in through SPARK-24891 (was: It's a class name typo c

[jira] [Created] (SPARK-25063) Rename class KnowNotNull to KnownNotNull

2018-08-08 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-25063: --- Summary: Rename class KnowNotNull to KnownNotNull Key: SPARK-25063 URL: https://issues.apache.org/jira/browse/SPARK-25063 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-25013) JDBC urls with jdbc:mariadb don't work as expected

2018-08-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574259#comment-16574259 ] Takeshi Yamamuro commented on SPARK-25013: -- yea, we can by registering a new di

[jira] [Resolved] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-08-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23596. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21535 [https://gi

[jira] [Assigned] (SPARK-23596) Modify Dataset test harness to include interpreted execution

2018-08-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23596: Assignee: Liang-Chi Hsieh > Modify Dataset test harness to include interpreted execution

[jira] [Updated] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-08-08 Thread andrzej.stankev...@gmail.com (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] andrzej.stankev...@gmail.com updated SPARK-25062: - Description: When Spark lists collection of files it does it on

[jira] [Updated] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-08-08 Thread andrzej.stankev...@gmail.com (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] andrzej.stankev...@gmail.com updated SPARK-25062: - Description: When Spark lists collection of files it does it on

[jira] [Updated] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-08-08 Thread andrzej.stankev...@gmail.com (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] andrzej.stankev...@gmail.com updated SPARK-25062: - Description: When Spark lists collection of files it does it on

[jira] [Created] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-08-08 Thread andrzej.stankev...@gmail.com (JIRA)
andrzej.stankev...@gmail.com created SPARK-25062: Summary: Clean up BlockLocations in FileStatus objects Key: SPARK-25062 URL: https://issues.apache.org/jira/browse/SPARK-25062 Project:

[jira] [Comment Edited] (SPARK-25051) where clause on dataset gives AnalysisException

2018-08-08 Thread MIK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573898#comment-16573898 ] MIK edited comment on SPARK-25051 at 8/8/18 9:32 PM: - df1 and df2, b

[jira] [Commented] (SPARK-25060) PySpark UDF in case statement is always run

2018-08-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573914#comment-16573914 ] Ryan Blue commented on SPARK-25060: --- I don't think that's quite the same thing since t

[jira] [Commented] (SPARK-25051) where clause on dataset gives AnalysisException

2018-08-08 Thread MIK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573915#comment-16573915 ] MIK commented on SPARK-25051: - I have added a sample program which shows this issue: Works

[jira] [Commented] (SPARK-25061) Spark SQL Thrift Server fails to not pick up hiveconf passing parameter

2018-08-08 Thread Zineng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573911#comment-16573911 ] Zineng Yuan commented on SPARK-25061: - A further investigation starting from thrift

[jira] [Updated] (SPARK-25061) Spark SQL Thrift Server fails to not pick up hiveconf passing parameter

2018-08-08 Thread Zineng Yuan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zineng Yuan updated SPARK-25061: Description: Spark thrift server should use passing parameter value and overwrites the same conf

[jira] [Commented] (SPARK-25051) where clause on dataset gives AnalysisException

2018-08-08 Thread MIK (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573898#comment-16573898 ] MIK commented on SPARK-25051: - df1 and df2, both are reading from S3 files df1 = spark.read

[jira] [Created] (SPARK-25061) Spark SQL Thrift Server fails to not pick up hiveconf passing parameter

2018-08-08 Thread Zineng Yuan (JIRA)
Zineng Yuan created SPARK-25061: --- Summary: Spark SQL Thrift Server fails to not pick up hiveconf passing parameter Key: SPARK-25061 URL: https://issues.apache.org/jira/browse/SPARK-25061 Project: Spark

[jira] [Comment Edited] (SPARK-25060) PySpark UDF in case statement is always run

2018-08-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573866#comment-16573866 ] Bryan Cutler edited comment on SPARK-25060 at 8/8/18 9:03 PM:

[jira] [Commented] (SPARK-25060) PySpark UDF in case statement is always run

2018-08-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573866#comment-16573866 ] Bryan Cutler commented on SPARK-25060: -- I believe this was brought up here https:/

[jira] [Commented] (SPARK-25060) PySpark UDF in case statement is always run

2018-08-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573834#comment-16573834 ] Ryan Blue commented on SPARK-25060: --- [~bryanc], [~holdenkarau], any ideas for how to f

[jira] [Created] (SPARK-25060) PySpark UDF in case statement is always run

2018-08-08 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-25060: - Summary: PySpark UDF in case statement is always run Key: SPARK-25060 URL: https://issues.apache.org/jira/browse/SPARK-25060 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25059) Exception while executing an action on DataFrame that read Json

2018-08-08 Thread Kunal Goswami (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Goswami updated SPARK-25059: -- External issue URL: https://issues.apache.org/jira/browse/SPARK-16845 > Exception while execut

[jira] [Updated] (SPARK-25059) Exception while executing an action on DataFrame that read Json

2018-08-08 Thread Kunal Goswami (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Goswami updated SPARK-25059: -- External issue URL: (was: https://issues.apache.org/jira/browse/SPARK-16845) > Exception wh

[jira] [Created] (SPARK-25059) Exception while executing an action on DataFrame that read Json

2018-08-08 Thread Kunal Goswami (JIRA)
Kunal Goswami created SPARK-25059: - Summary: Exception while executing an action on DataFrame that read Json Key: SPARK-25059 URL: https://issues.apache.org/jira/browse/SPARK-25059 Project: Spark

[jira] [Commented] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573785#comment-16573785 ] Apache Spark commented on SPARK-19851: -- User 'dilipbiswal' has created a pull reque

[jira] [Assigned] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23940: Assignee: Apache Spark > High-order function: transform_values(map, function) → > map >

[jira] [Assigned] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23940: Assignee: (was: Apache Spark) > High-order function: transform_values(map, function)

[jira] [Commented] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573764#comment-16573764 ] Apache Spark commented on SPARK-23940: -- User 'codeatri' has created a pull request

[jira] [Commented] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573747#comment-16573747 ] Apache Spark commented on SPARK-23939: -- User 'codeatri' has created a pull request

[jira] [Comment Edited] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

2018-08-08 Thread bharath kumar avusherla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573700#comment-16573700 ] bharath kumar avusherla edited comment on SPARK-23050 at 8/8/18 6:51 PM: -

[jira] [Commented] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

2018-08-08 Thread bharath kumar avusherla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573700#comment-16573700 ] bharath kumar avusherla commented on SPARK-23050: - Is there any way we c

[jira] [Commented] (SPARK-23912) High-order function: array_distinct(x) → array

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573699#comment-16573699 ] Apache Spark commented on SPARK-23912: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-24251) DataSourceV2: Add AppendData logical operation

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573684#comment-16573684 ] Apache Spark commented on SPARK-24251: -- User 'rdblue' has created a pull request fo

[jira] [Assigned] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25005: Assignee: (was: Apache Spark) > Structured streaming doesn't support kafka transactio

[jira] [Commented] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573585#comment-16573585 ] Apache Spark commented on SPARK-25005: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25005: Assignee: Apache Spark > Structured streaming doesn't support kafka transaction (creating

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-08 Thread Lukas Rytz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573570#comment-16573570 ] Lukas Rytz commented on SPARK-25044: This problem relates to what had to be solved f

[jira] [Assigned] (SPARK-25058) Use Block.isEmpty/nonEmpty to check whether the code is empty or not.

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25058: Assignee: (was: Apache Spark) > Use Block.isEmpty/nonEmpty to check whether the code

[jira] [Assigned] (SPARK-25058) Use Block.isEmpty/nonEmpty to check whether the code is empty or not.

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25058: Assignee: Apache Spark > Use Block.isEmpty/nonEmpty to check whether the code is empty or

[jira] [Commented] (SPARK-25058) Use Block.isEmpty/nonEmpty to check whether the code is empty or not.

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573526#comment-16573526 ] Apache Spark commented on SPARK-25058: -- User 'ueshin' has created a pull request fo

[jira] [Created] (SPARK-25058) Use Block.isEmpty/nonEmpty to check whether the code is empty or not.

2018-08-08 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-25058: - Summary: Use Block.isEmpty/nonEmpty to check whether the code is empty or not. Key: SPARK-25058 URL: https://issues.apache.org/jira/browse/SPARK-25058 Project: Spar

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573513#comment-16573513 ] Sean Owen commented on SPARK-25044: --- Yep, that's the crux of the issue here – not clea

[jira] [Commented] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573508#comment-16573508 ] Apache Spark commented on SPARK-25036: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573503#comment-16573503 ] Sean Owen commented on SPARK-25029: --- I don't think it'll come to that; looks like some

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-08 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573495#comment-16573495 ] shane knapp commented on SPARK-25029: - i really don't like the thought of resorting

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573493#comment-16573493 ] Wenchen Fan commented on SPARK-25044: - Sorry I've not read all the comments here, bu

[jira] [Created] (SPARK-25057) Unable to start spark on master URL

2018-08-08 Thread Shivam Gupta (JIRA)
Shivam Gupta created SPARK-25057: Summary: Unable to start spark on master URL Key: SPARK-25057 URL: https://issues.apache.org/jira/browse/SPARK-25057 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573400#comment-16573400 ] Sean Owen commented on SPARK-25044: --- I dug into this, and found that you can't directl

[jira] [Assigned] (SPARK-25056) Unify the InConversion and BinaryComparison behaviour when InConversion's list only contains one datatype

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25056: Assignee: Apache Spark > Unify the InConversion and BinaryComparison behaviour when InCon

[jira] [Assigned] (SPARK-25056) Unify the InConversion and BinaryComparison behaviour when InConversion's list only contains one datatype

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25056: Assignee: (was: Apache Spark) > Unify the InConversion and BinaryComparison behaviour

[jira] [Commented] (SPARK-25056) Unify the InConversion and BinaryComparison behaviour when InConversion's list only contains one datatype

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573394#comment-16573394 ] Apache Spark commented on SPARK-25056: -- User 'wangyum' has created a pull request f

[jira] [Created] (SPARK-25056) Unify the InConversion and BinaryComparison behaviour when InConversion's list only contains one datatype

2018-08-08 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25056: --- Summary: Unify the InConversion and BinaryComparison behaviour when InConversion's list only contains one datatype Key: SPARK-25056 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Commented] (SPARK-25031) The schema of MapType can not be printed correctly

2018-08-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573299#comment-16573299 ] Marco Gaido commented on SPARK-25031: - [~smilegator] shall this be resolved as http

[jira] [Assigned] (SPARK-24774) support reading AVRO logical types - Decimal

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24774: Assignee: Apache Spark > support reading AVRO logical types - Decimal > -

[jira] [Commented] (SPARK-24774) support reading AVRO logical types - Decimal

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573235#comment-16573235 ] Apache Spark commented on SPARK-24774: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-24774) support reading AVRO logical types - Decimal

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24774: Assignee: (was: Apache Spark) > support reading AVRO logical types - Decimal > --

[jira] [Updated] (SPARK-24774) support reading AVRO logical types - Decimal

2018-08-08 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-24774: --- Summary: support reading AVRO logical types - Decimal (was: support reading AVRO logical ty

[jira] [Updated] (SPARK-24772) support reading AVRO logical types - Date

2018-08-08 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-24772: --- Summary: support reading AVRO logical types - Date (was: support reading AVRO logical types

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573221#comment-16573221 ] Thomas Graves commented on SPARK-24924: --- | There was a discussion about why we sho

[jira] [Commented] (SPARK-24502) flaky test: UnsafeRowSerializerSuite

2018-08-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573175#comment-16573175 ] Wenchen Fan commented on SPARK-24502: - Mostly you don't need to care about it. But w

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573160#comment-16573160 ] Sean Owen commented on SPARK-25044: --- I see, it sounds like the code needs to look for

[jira] [Commented] (SPARK-25047) Can't assign SerializedLambda to scala.Function1 in deserialization of BucketedRandomProjectionLSHModel

2018-08-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573158#comment-16573158 ] Sean Owen commented on SPARK-25047: --- Thanks [~lrytz] – sounds like it is at least outs

[jira] [Commented] (SPARK-25055) MessageWithHeader transfer ByteBuffer from Netty's CompositeByteBuf many times

2018-08-08 Thread DENG FEI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573154#comment-16573154 ] DENG FEI commented on SPARK-25055: -- {code:java} private int copyByteBuf(ByteBuf buf, Wr

[jira] [Commented] (SPARK-24502) flaky test: UnsafeRowSerializerSuite

2018-08-08 Thread Paul Praet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573149#comment-16573149 ] Paul Praet commented on SPARK-24502: [~cloud_fan] Does this mean users also have to

[jira] [Created] (SPARK-25055) MessageWithHeader transfer ByteBuffer from Netty's CompositeByteBuf many times

2018-08-08 Thread DENG FEI (JIRA)
DENG FEI created SPARK-25055: Summary: MessageWithHeader transfer ByteBuffer from Netty's CompositeByteBuf many times Key: SPARK-25055 URL: https://issues.apache.org/jira/browse/SPARK-25055 Project: Spark

[jira] [Assigned] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25028: Assignee: (was: Apache Spark) > AnalyzePartitionCommand failed with NPE if value is n

[jira] [Assigned] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25028: Assignee: Apache Spark > AnalyzePartitionCommand failed with NPE if value is null > -

[jira] [Commented] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573120#comment-16573120 ] Apache Spark commented on SPARK-25028: -- User 'mgaido91' has created a pull request

[jira] [Updated] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Izek Greenfield updated SPARK-25028: Description: on line 143: val partitionColumnValues = partitionColumns.indices.map(r.get(

[jira] [Reopened] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Izek Greenfield reopened SPARK-25028: - Add sample code > AnalyzePartitionCommand failed with NPE if value is null > --

[jira] [Updated] (SPARK-25028) AnalyzePartitionCommand failed with NPE if value is null

2018-08-08 Thread Izek Greenfield (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Izek Greenfield updated SPARK-25028: Description: on line 143: val partitionColumnValues = partitionColumns.indices.map(r.get(

[jira] [Commented] (SPARK-23911) High-order function: aggregate(array, initialState S, inputFunction, outputFunction) → R

2018-08-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572871#comment-16572871 ] Apache Spark commented on SPARK-23911: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-24918) Executor Plugin API

2018-08-08 Thread Luca Canali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572850#comment-16572850 ] Luca Canali commented on SPARK-24918: - [~irashid] I agree that the proposal should b