[jira] [Updated] (SPARK-27637) If exception occured while fetching blocks by netty block transfer service, check whether the relative executor is alive before retry

2019-05-12 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27637: Component/s: (was: Block Manager) > If exception occured while fetching blocks by netty block

[jira] [Updated] (SPARK-27637) If exception occured while fetching blocks by netty block transfer service, check whether the relative executor is alive before retry

2019-05-12 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27637: Affects Version/s: (was: 2.4.2) (was: 2.3.2) 2.3.3

[jira] [Updated] (SPARK-27637) If exception occured while fetching blocks by netty block transfer service, check whether the relative executor is alive before retry

2019-05-12 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27637: Component/s: Shuffle > If exception occured while fetching blocks by netty block transfer service, >

[jira] [Updated] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

2019-05-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27685: --- Labels: correctness (was: ) > `union` doesn't promote non-nullable columns of struct to nullable >

[jira] [Updated] (SPARK-27686) Update migration guide

2019-05-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27686: Summary: Update migration guide (was: Update) > Update migration guide >

[jira] [Updated] (SPARK-27686) Update migration guide

2019-05-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27686: Description: The built-in Hive 2.3 fixes the following issues: * HIVE-6727: Table level stats

[jira] [Updated] (SPARK-27686) Update migration guide

2019-05-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27686: Description: The built-in Hive 2.3 fixes the following issues: * HIVE-6727: Table level stats

[jira] [Created] (SPARK-27686) Update

2019-05-12 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27686: --- Summary: Update Key: SPARK-27686 URL: https://issues.apache.org/jira/browse/SPARK-27686 Project: Spark Issue Type: Sub-task Components:

[jira] [Updated] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

2019-05-12 Thread Huon Wilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huon Wilson updated SPARK-27685: Description: When doing a {{union}} of two dataframes, a column that is nullable in one of the

[jira] [Created] (SPARK-27685) `union` doesn't promote non-nullable columns of struct to nullable

2019-05-12 Thread Huon Wilson (JIRA)
Huon Wilson created SPARK-27685: --- Summary: `union` doesn't promote non-nullable columns of struct to nullable Key: SPARK-27685 URL: https://issues.apache.org/jira/browse/SPARK-27685 Project: Spark

[jira] [Commented] (SPARK-27335) cannot collect() from Correlation.corr

2019-05-12 Thread Michael Chirico (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838241#comment-16838241 ] Michael Chirico commented on SPARK-27335: - Also came across this in another context on Spark

[jira] [Commented] (SPARK-23191) Workers registration failes in case of network drop

2019-05-12 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838237#comment-16838237 ] wuyi commented on SPARK-23191: -- Hi [~neeraj20gupta]  Can you explain more about the part of _running

[jira] [Commented] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838229#comment-16838229 ] Kazuaki Ishizaki commented on SPARK-27684: -- Interesting idea > Reduce ScalaUDF conversion

[jira] [Commented] (SPARK-21367) R older version of Roxygen2 on Jenkins

2019-05-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838188#comment-16838188 ] Felix Cheung commented on SPARK-21367: -- great thx! > R older version of Roxygen2 on Jenkins >

[jira] [Commented] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838187#comment-16838187 ] Felix Cheung commented on SPARK-27684: -- definitely could be interesting.. > Reduce ScalaUDF

[jira] [Updated] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27684: --- Description: I believe that we can reduce ScalaUDF overheads when operating over primitive types.

[jira] [Updated] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27684: --- Description: I believe that we can reduce ScalaUDF overheads when operating over primitive types.

[jira] [Updated] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27684: --- Description: I believe that we can reduce ScalaUDF overheads when operating over primitive types.

[jira] [Updated] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-27684: --- Description: I believe that we can reduce ScalaUDF overheads when operating over primitive types.

[jira] [Created] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-27684: -- Summary: Reduce ScalaUDF conversion overheads for primitives Key: SPARK-27684 URL: https://issues.apache.org/jira/browse/SPARK-27684 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-27343) Use ConfigEntry for hardcoded configs for spark-sql-kafka

2019-05-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27343. --- Resolution: Fixed Fix Version/s: 3.1.0 Issue resolved by pull request 24574

[jira] [Updated] (SPARK-27343) Use ConfigEntry for hardcoded configs for spark-sql-kafka

2019-05-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27343: -- Fix Version/s: (was: 3.1.0) 3.0.0 > Use ConfigEntry for hardcoded configs for

[jira] [Assigned] (SPARK-27343) Use ConfigEntry for hardcoded configs for spark-sql-kafka

2019-05-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27343: - Assignee: hehuiyuan > Use ConfigEntry for hardcoded configs for spark-sql-kafka >

[jira] [Commented] (SPARK-27683) Remove usage of TraversableOnce

2019-05-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838092#comment-16838092 ] Sean Owen commented on SPARK-27683: --- Yeah I mentioned that above -- I don't think it helps as it still

[jira] [Updated] (SPARK-27074) Hive 3.1 metastore support HiveClientImpl.runHive

2019-05-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Summary: Hive 3.1 metastore support HiveClientImpl.runHive (was: Refactor HiveClientImpl

[jira] [Comment Edited] (SPARK-27683) Remove usage of TraversableOnce

2019-05-12 Thread PJ Fanning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838086#comment-16838086 ] PJ Fanning edited comment on SPARK-27683 at 5/12/19 3:01 PM: - [~srowen]

[jira] [Commented] (SPARK-27683) Remove usage of TraversableOnce

2019-05-12 Thread PJ Fanning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838086#comment-16838086 ] PJ Fanning commented on SPARK-27683: [~srowen] would it be possible to use the

[jira] [Resolved] (SPARK-27675) do not use MutableColumnarRow in ColumnarBatch

2019-05-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27675. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24581