[jira] [Commented] (SPARK-30983) Support more than 5 typed column in typed Dataset.select API

2020-02-27 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047278#comment-17047278 ] L. C. Hsieh commented on SPARK-30983: - cc [~cloud_fan] > Support more than 5 typed column in typed

[jira] [Created] (SPARK-30983) Support more than 5 typed column in typed Dataset.select API

2020-02-27 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-30983: --- Summary: Support more than 5 typed column in typed Dataset.select API Key: SPARK-30983 URL: https://issues.apache.org/jira/browse/SPARK-30983 Project: Spark

[jira] [Updated] (SPARK-30982) List All the removed APIs of Spark SQL and Core

2020-02-27 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30982: - Attachment: sql_signature.diff added_sql_class

[jira] [Updated] (SPARK-30982) List All the removed APIs of Spark SQL and Core

2020-02-27 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30982: - Attachment: core_signature.diff added_core_class

[jira] [Assigned] (SPARK-30982) List All the removed APIs of Spark SQL and Core

2020-02-27 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-30982: --- Assignee: zhengruifeng (was: Xiao Li) > List All the removed APIs of Spark SQL and Core >

[jira] [Created] (SPARK-30982) List All the removed APIs of Spark SQL and Core

2020-02-27 Thread Xiao Li (Jira)
Xiao Li created SPARK-30982: --- Summary: List All the removed APIs of Spark SQL and Core Key: SPARK-30982 URL: https://issues.apache.org/jira/browse/SPARK-30982 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-30981) Fix flaky "Test basic decommissioning" test

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047270#comment-17047270 ] Dongjoon Hyun commented on SPARK-30981: --- Could you take a look at this, [~holden]? > Fix flaky

[jira] [Assigned] (SPARK-30981) Fix flaky "Test basic decommissioning" test

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30981: - Assignee: (was: Dongjoon Hyun) > Fix flaky "Test basic decommissioning" test >

[jira] [Created] (SPARK-30981) Fix flaky "Test basic decommissioning" test

2020-02-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-30981: - Summary: Fix flaky "Test basic decommissioning" test Key: SPARK-30981 URL: https://issues.apache.org/jira/browse/SPARK-30981 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25474) Update the docs for spark.sql.statistics.fallBackToHdfs

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25474: -- Summary: Update the docs for spark.sql.statistics.fallBackToHdfs (was: Support

[jira] [Resolved] (SPARK-30902) default table provider should be decided by catalog implementations

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30902. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27650

[jira] [Closed] (SPARK-26599) BroardCast hint can not work with PruneFileSourcePartitions

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26599. - > BroardCast hint can not work with PruneFileSourcePartitions >

[jira] [Updated] (SPARK-26599) BroardCast hint can not work with PruneFileSourcePartitions

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26599: -- Issue Type: Bug (was: Improvement) > BroardCast hint can not work with

[jira] [Updated] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

2020-02-27 Thread Pradyumn Agrawal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradyumn Agrawal updated SPARK-30980: - Shepherd: Apache Spark > Issue not resolved of Caught Hive MetaException attempting to

[jira] [Updated] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

2020-02-27 Thread Pradyumn Agrawal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradyumn Agrawal updated SPARK-30980: - Description: I am querying on table created in Hive. Getting repetitive exception of

[jira] [Created] (SPARK-30980) Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive

2020-02-27 Thread Pradyumn Agrawal (Jira)
Pradyumn Agrawal created SPARK-30980: Summary: Issue not resolved of Caught Hive MetaException attempting to get partition metadata by filter from Hive Key: SPARK-30980 URL:

[jira] [Resolved] (SPARK-30972) PruneHiveTablePartitions should be executed as earlyScanPushDownRules

2020-02-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30972. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27723

[jira] [Assigned] (SPARK-30972) PruneHiveTablePartitions should be executed as earlyScanPushDownRules

2020-02-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30972: --- Assignee: wuyi > PruneHiveTablePartitions should be executed as earlyScanPushDownRules >

[jira] [Resolved] (SPARK-30681) Add higher order functions API to PySpark

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30681. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27406

[jira] [Assigned] (SPARK-30682) Add higher order functions API to SparkR

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30682: Assignee: Maciej Szymkiewicz > Add higher order functions API to SparkR >

[jira] [Resolved] (SPARK-30682) Add higher order functions API to SparkR

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30682. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27433

[jira] [Assigned] (SPARK-30681) Add higher order functions API to PySpark

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30681: Assignee: Maciej Szymkiewicz > Add higher order functions API to PySpark >

[jira] [Resolved] (SPARK-30955) Exclude Generate output when aliasing in nested column pruning

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30955. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27702

[jira] [Resolved] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2020-02-27 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-14643. - Resolution: Won't Fix > Remove overloaded methods which become ambiguous in Scala 2.12 >

[jira] [Commented] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047170#comment-17047170 ] Hyukjin Kwon commented on SPARK-26836: -- I am lowering the priority to Critical as it's at least not

[jira] [Updated] (SPARK-26836) Columns get switched in Spark SQL using Avro backed Hive table if schema evolves

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26836: - Priority: Critical (was: Blocker) > Columns get switched in Spark SQL using Avro backed Hive

[jira] [Updated] (SPARK-30979) spark-submit - no need to resolve dependencies in kubernetes cluster mode

2020-02-27 Thread Dyno (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated SPARK-30979: - Summary: spark-submit - no need to resolve dependencies in kubernetes cluster mode (was: no need to resolve

[jira] [Created] (SPARK-30979) no need to resolve dependencies in kubernetes cluster mode

2020-02-27 Thread Dyno (Jira)
Dyno created SPARK-30979: Summary: no need to resolve dependencies in kubernetes cluster mode Key: SPARK-30979 URL: https://issues.apache.org/jira/browse/SPARK-30979 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-30968) Upgrade aws-java-sdk-sts to 1.11.655

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30968: - Assignee: Dongjoon Hyun > Upgrade aws-java-sdk-sts to 1.11.655 >

[jira] [Resolved] (SPARK-30968) Upgrade aws-java-sdk-sts to 1.11.655

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30968. --- Fix Version/s: 3.0.0 2.4.6 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047029#comment-17047029 ] Dongjoon Hyun commented on SPARK-30855: --- According to the public plan, no. Probably, 3.0.0 RC1? -

[jira] [Comment Edited] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047029#comment-17047029 ] Dongjoon Hyun edited comment on SPARK-30855 at 2/27/20 11:00 PM: -

[jira] [Commented] (SPARK-30442) Write mode ignored when using CodecStreams

2020-02-27 Thread Abhishek Madav (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047024#comment-17047024 ] Abhishek Madav commented on SPARK-30442: In case of task failures, say the task fails to write

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2020-02-27 Thread Jorge Machado (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046958#comment-17046958 ] Jorge Machado commented on SPARK-26412: --- Thanks for the Tipp. It helps > Allow Pandas UDF to

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-27 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/27/20 7:55 PM: -    I audited

[jira] [Commented] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046929#comment-17046929 ] Xingbo Jiang commented on SPARK-30969: -- I created https://issues.apache.org/jira/browse/SPARK-30978

[jira] [Updated] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

2020-02-27 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-30978: - Description: Based on our experience, there is no scenario that necessarily requires deploying

[jira] [Created] (SPARK-30978) Remove multiple workers on the same host support from Standalone backend

2020-02-27 Thread Xingbo Jiang (Jira)
Xingbo Jiang created SPARK-30978: Summary: Remove multiple workers on the same host support from Standalone backend Key: SPARK-30978 URL: https://issues.apache.org/jira/browse/SPARK-30978 Project:

[jira] [Updated] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

2020-02-27 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-30977: -- Target Version/s: 3.0.0 > ResourceProfile and Builder should be private in spark 3.0 >

[jira] [Assigned] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

2020-02-27 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-30977: - Assignee: (was: Thomas Graves) > ResourceProfile and Builder should be private in

[jira] [Commented] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

2020-02-27 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046891#comment-17046891 ] Thomas Graves commented on SPARK-30977: --- I'm working on this should have pr by end of day >

[jira] [Created] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

2020-02-27 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30977: - Summary: ResourceProfile and Builder should be private in spark 3.0 Key: SPARK-30977 URL: https://issues.apache.org/jira/browse/SPARK-30977 Project: Spark

[jira] [Deleted] (SPARK-30976) Improve Maven Install Logic in build/mvn

2020-02-27 Thread Yin Huai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai deleted SPARK-30976: - > Improve Maven Install Logic in build/mvn > > >

[jira] [Updated] (SPARK-30976) Improve Maven Install Logic in build/mvn

2020-02-27 Thread Wesley Hsiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley Hsiao updated SPARK-30976: - Description: The current code at lacks a validation step to test the installed maven binary at

[jira] [Created] (SPARK-30976) Improve Maven Install Logic in build/mvn

2020-02-27 Thread Wesley Hsiao (Jira)
Wesley Hsiao created SPARK-30976: Summary: Improve Maven Install Logic in build/mvn Key: SPARK-30976 URL: https://issues.apache.org/jira/browse/SPARK-30976 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-02-27 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-30951: -- Description: tl;dr: We recently discovered some Spark 2.x sites that have lots of data

[jira] [Commented] (SPARK-30961) Arrow enabled: to_pandas with date column fails

2020-02-27 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046801#comment-17046801 ] Bryan Cutler commented on SPARK-30961: -- Yes, we should be able to keep Spark 3.x up to date with

[jira] [Commented] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046795#comment-17046795 ] Xiangrui Meng commented on SPARK-30969: --- [~Ngone51] [~jiangxb1987] Is there a JIRA to deprecate

[jira] [Updated] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30969: -- Environment: (was: Resource coordination is used for the case where multiple workers

[jira] [Updated] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30969: -- Priority: Critical (was: Major) > Remove resource coordination support from Standalone >

[jira] [Assigned] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-30969: - Assignee: wuyi > Remove resource coordination support from Standalone >

[jira] [Updated] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30969: -- Description: Resource coordination is used for the case where multiple workers running on the

[jira] [Updated] (SPARK-30975) Rename config for spark.<>.memoryOverhead

2020-02-27 Thread Miquel Angel Andreu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miquel Angel Andreu updated SPARK-30975: Summary: Rename config for spark.<>.memoryOverhead (was: Rename config to

[jira] [Created] (SPARK-30975) Rename config to spark.executor.memoryOverhead

2020-02-27 Thread Miquel Angel Andreu (Jira)
Miquel Angel Andreu created SPARK-30975: --- Summary: Rename config to spark.executor.memoryOverhead Key: SPARK-30975 URL: https://issues.apache.org/jira/browse/SPARK-30975 Project: Spark

[jira] [Commented] (SPARK-28994) Document working of Adaptive

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046710#comment-17046710 ] Takeshi Yamamuro commented on SPARK-28994: -- Adaptive? This means adaptive execution? btw, is it

[jira] [Commented] (SPARK-28993) Document Working of Bucketing

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046708#comment-17046708 ] Takeshi Yamamuro commented on SPARK-28993: -- Any update? btw, is it worth documenting this in

[jira] [Commented] (SPARK-28965) Document workings of CBO

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046706#comment-17046706 ] Takeshi Yamamuro commented on SPARK-28965: -- Any update? btw, is it worth documenting this in

[jira] [Resolved] (SPARK-28995) Document working of Spark Streaming

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28995. -- Resolution: Invalid > Document working of Spark Streaming >

[jira] [Commented] (SPARK-28995) Document working of Spark Streaming

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046704#comment-17046704 ] Takeshi Yamamuro commented on SPARK-28995: -- I think this is not related to the SQL refs, so

[jira] [Commented] (SPARK-29458) Document scalar functions usage in APIs in SQL getting started.

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046701#comment-17046701 ] Takeshi Yamamuro commented on SPARK-29458: -- [~dkbiswal] Any update? > Document scalar

[jira] [Commented] (SPARK-30095) create function syntax has to be enhance in Doc for multiple dependent jars

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046698#comment-17046698 ] Takeshi Yamamuro commented on SPARK-30095: -- [~abhishek.akg] Any update? > create function

[jira] [Resolved] (SPARK-30635) Document PARTITIONED BY Clause of CREATE statement in SQL Reference

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30635. -- Resolution: Duplicate I think this has been resolved by SPARK-28794 

[jira] [Resolved] (SPARK-30693) Document STORED AS Clause of CREATE statement in SQL Reference

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30693. -- Resolution: Duplicate I think this has been resolved by SPARK-28794

[jira] [Commented] (SPARK-30635) Document PARTITIONED BY Clause of CREATE statement in SQL Reference

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046690#comment-17046690 ] Takeshi Yamamuro commented on SPARK-30635: -- [~jobitmathew] still working on it? > Document

[jira] [Updated] (SPARK-30974) org.apache.spark.sql.AnalysisException: expression 'default.udfvalidation.`empname`' is neither present in the group by, nor is it an aggregate function.

2020-02-27 Thread Akshay (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshay updated SPARK-30974: --- Summary: org.apache.spark.sql.AnalysisException: expression 'default.udfvalidation.`empname`' is neither

[jira] [Created] (SPARK-30974) org.apache.spark.sql.AnalysisException: expression 'default.udfvalidation.`empname`' is neither present in the group by, nor is it an aggregate function. Add to group by

2020-02-27 Thread Akshay (Jira)
Akshay created SPARK-30974: -- Summary: org.apache.spark.sql.AnalysisException: expression 'default.udfvalidation.`empname`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you

[jira] [Comment Edited] (SPARK-29969) parse_url function result in incorrect result

2020-02-27 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045059#comment-17045059 ] YoungGyu Chun edited comment on SPARK-29969 at 2/27/20 2:16 PM:

[jira] [Resolved] (SPARK-30956) Use intercept instead of try-catch to assert failures in IntervalUtilsSuite

2020-02-27 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30956. -- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-27 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046654#comment-17046654 ] Benoit Roy commented on SPARK-30855: Ok thanks for letting me know.  Are there any plan for another

[jira] [Resolved] (SPARK-30937) Migration guide for Hive 2.3

2020-02-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30937. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27670

[jira] [Assigned] (SPARK-30937) Migration guide for Hive 2.3

2020-02-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30937: --- Assignee: wuyi > Migration guide for Hive 2.3 > > >

[jira] [Created] (SPARK-30973) ScriptTransformationExec should wait for the termination of process when scriptOutputReader hasNext return false

2020-02-27 Thread Sun Ke (Jira)
Sun Ke created SPARK-30973: -- Summary: ScriptTransformationExec should wait for the termination of process when scriptOutputReader hasNext return false Key: SPARK-30973 URL:

[jira] [Created] (SPARK-30972) PruneHiveTablePartitions should be executed as earlyScanPushDownRules

2020-02-27 Thread wuyi (Jira)
wuyi created SPARK-30972: Summary: PruneHiveTablePartitions should be executed as earlyScanPushDownRules Key: SPARK-30972 URL: https://issues.apache.org/jira/browse/SPARK-30972 Project: Spark Issue

[jira] [Resolved] (SPARK-30971) Support MySQL Kerberos login in JDBC connector

2020-02-27 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-30971. --- Resolution: Won't Do > Support MySQL Kerberos login in JDBC connector >

[jira] [Commented] (SPARK-30971) Support MySQL Kerberos login in JDBC connector

2020-02-27 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046550#comment-17046550 ] Gabor Somogyi commented on SPARK-30971: --- Just for the record I've created this jira since MySQL

[jira] [Created] (SPARK-30971) Support MySQL Kerberos login in JDBC connector

2020-02-27 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-30971: - Summary: Support MySQL Kerberos login in JDBC connector Key: SPARK-30971 URL: https://issues.apache.org/jira/browse/SPARK-30971 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046542#comment-17046542 ] Hyukjin Kwon edited comment on SPARK-26412 at 2/27/20 12:03 PM: You

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046542#comment-17046542 ] Hyukjin Kwon commented on SPARK-26412: -- You cannot separate one iterator to multiple iterators

[jira] [Commented] (SPARK-30970) Fix NPE in resolving k8s master url

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046454#comment-17046454 ] Dongjoon Hyun commented on SPARK-30970: --- BTW, [~Qin Yao]. Could you check 2.3.4 behavior and

[jira] [Commented] (SPARK-30970) Fix NPE in resolving k8s master url

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046453#comment-17046453 ] Dongjoon Hyun commented on SPARK-30970: --- Since the root cause is the user mistakes, the prevention

[jira] [Updated] (SPARK-30970) Fix NPE in resolving k8s master url

2020-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30970: -- Priority: Minor (was: Major) > Fix NPE in resolving k8s master url >

[jira] [Updated] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

2020-02-27 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30929: - Environment: (was: Audit new public Scala APIs added to MLlib & GraphX. Take note of: *

[jira] [Updated] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

2020-02-27 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30929: - Description: Audit new public Scala APIs added to MLlib & GraphX. Take note of: *

[jira] [Comment Edited] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-27 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046228#comment-17046228 ] zhengruifeng edited comment on SPARK-30932 at 2/27/20 9:47 AM: --- I checked

[jira] [Created] (SPARK-30970) Fix NPE in resolving k8s master url

2020-02-27 Thread Kent Yao (Jira)
Kent Yao created SPARK-30970: Summary: Fix NPE in resolving k8s master url Key: SPARK-30970 URL: https://issues.apache.org/jira/browse/SPARK-30970 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-30969) Remove resource coordination support from Standalone

2020-02-27 Thread wuyi (Jira)
wuyi created SPARK-30969: Summary: Remove resource coordination support from Standalone Key: SPARK-30969 URL: https://issues.apache.org/jira/browse/SPARK-30969 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2020-02-27 Thread Jorge Machado (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046373#comment-17046373 ] Jorge Machado commented on SPARK-26412: --- Well I was thinking on something more. like I would like

[jira] [Commented] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames

2020-02-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046365#comment-17046365 ] Hyukjin Kwon commented on SPARK-26412: -- You can do it via: {code} def map_func(batch_iter):

[jira] [Commented] (SPARK-29969) parse_url function result in incorrect result

2020-02-27 Thread Victor Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046359#comment-17046359 ] Victor Zhang commented on SPARK-29969: -- [~younggyuchun]  The description may be a bit confusing. I

[jira] [Commented] (SPARK-30961) Arrow enabled: to_pandas with date column fails

2020-02-27 Thread Nicolas Renkamp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046337#comment-17046337 ] Nicolas Renkamp commented on SPARK-30961: - [~bryanc] Thanks for the quick reply.  Actually, I