[jira] [Updated] (SPARK-29861) Reduce downtime in Spark standalone HA master switch

2019-11-12 Thread Robin Wolters (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Wolters updated SPARK-29861: -- Summary: Reduce downtime in Spark standalone HA master switch (was: Reduce leader election

[jira] [Created] (SPARK-29872) Improper cache strategy in examples

2019-11-12 Thread Dong Wang (Jira)
Dong Wang created SPARK-29872: - Summary: Improper cache strategy in examples Key: SPARK-29872 URL: https://issues.apache.org/jira/browse/SPARK-29872 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-29869) HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29869: Description: {noformat} scala> spark.table("hive_table").show java.lang.AssertionError: assertion

[jira] [Created] (SPARK-29871) Flaky test: ImageFileFormatTest.test_read_images

2019-11-12 Thread wuyi (Jira)
wuyi created SPARK-29871: Summary: Flaky test: ImageFileFormatTest.test_read_images Key: SPARK-29871 URL: https://issues.apache.org/jira/browse/SPARK-29871 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-29390) Add the justify_days(), justify_hours() and justify_interval() functions

2019-11-12 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29390. -- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

2019-11-12 Thread Jalpan Randeri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973023#comment-16973023 ] Jalpan Randeri commented on SPARK-25351: Yes, I will take up this changes as part of my PR >

[jira] [Commented] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

2019-11-12 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973007#comment-16973007 ] ABHISHEK KUMAR GUPTA commented on SPARK-29854: -- [~hyukjin.kwon]: Hi I checked in Hive

[jira] [Reopened] (SPARK-29462) The data type of "array()" should be array

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-29462: -- Assignee: (was: Aman Omer) > The data type of "array()" should be array >

[jira] [Updated] (SPARK-29462) The data type of "array()" should be array

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29462: - Fix Version/s: (was: 3.0.0) > The data type of "array()" should be array >

[jira] [Commented] (SPARK-29462) The data type of "array()" should be array

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973000#comment-16973000 ] Hyukjin Kwon commented on SPARK-29462: -- Reverted due to the reasons

[jira] [Commented] (SPARK-29870) Unify the logic of multi-units interval string to CalendarInterval

2019-11-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972992#comment-16972992 ] Kent Yao commented on SPARK-29870: -- working on this > Unify the logic of multi-units interval string

[jira] [Created] (SPARK-29870) Unify the logic of multi-units interval string to CalendarInterval

2019-11-12 Thread Kent Yao (Jira)
Kent Yao created SPARK-29870: Summary: Unify the logic of multi-units interval string to CalendarInterval Key: SPARK-29870 URL: https://issues.apache.org/jira/browse/SPARK-29870 Project: Spark

[jira] [Commented] (SPARK-29855) typed literals with negative sign with proper result or exception

2019-11-12 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972983#comment-16972983 ] Kent Yao commented on SPARK-29855: -- This is the related ticket that involves the negative sign before

[jira] [Updated] (SPARK-29869) HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29869: Description: {noformat} scala> spark.table("hive_table").show java.lang.AssertionError: assertion

[jira] [Commented] (SPARK-29868) Add a benchmark for Adaptive Execution

2019-11-12 Thread leonard ding (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972974#comment-16972974 ] leonard ding commented on SPARK-29868: -- I will pick it > Add a benchmark for Adaptive Execution >

[jira] [Updated] (SPARK-29869) HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29869: Attachment: SPARK-29869.jpg > HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError

[jira] [Created] (SPARK-29869) HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError

2019-11-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29869: --- Summary: HiveMetastoreCatalog#convertToLogicalRelation throws AssertionError Key: SPARK-29869 URL: https://issues.apache.org/jira/browse/SPARK-29869 Project: Spark

[jira] [Commented] (SPARK-29868) Add a benchmark for Adaptive Execution

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972967#comment-16972967 ] Yuming Wang commented on SPARK-29868: - Examples: 

[jira] [Updated] (SPARK-29868) Add a benchmark for Adaptive Execution

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29868: Description: Add a benchmark for Adaptive Execution to evaluate SortMergeJoin to  BroadcastJoin

[jira] [Updated] (SPARK-29868) Add a benchmark for Adaptive Execution

2019-11-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29868: Attachment: BroadcastJoin.jpeg SortMergeJoin.jpeg > Add a benchmark for Adaptive

[jira] [Created] (SPARK-29868) Add a benchmark for Adaptive Execution

2019-11-12 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29868: --- Summary: Add a benchmark for Adaptive Execution Key: SPARK-29868 URL: https://issues.apache.org/jira/browse/SPARK-29868 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-29396) Extend Spark plugin interface to driver

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29396. -- Fix Version/s: 3.0.0 Assignee: Marcelo Masiero Vanzin Resolution: Done >

[jira] [Resolved] (SPARK-29399) Remove old ExecutorPlugin API (or wrap it using new API)

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29399. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26390

[jira] [Assigned] (SPARK-29399) Remove old ExecutorPlugin API (or wrap it using new API)

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29399: Assignee: Marcelo Masiero Vanzin > Remove old ExecutorPlugin API (or wrap it using new

[jira] [Assigned] (SPARK-29570) Improve tooltip for Executor Tab for Shuffle Write,Blacklisted,Logs,Threaddump columns

2019-11-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29570: Assignee: Ankit Raj Boudh > Improve tooltip for Executor Tab for Shuffle >

[jira] [Resolved] (SPARK-29570) Improve tooltip for Executor Tab for Shuffle Write,Blacklisted,Logs,Threaddump columns

2019-11-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29570. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26263

[jira] [Resolved] (SPARK-29844) Improper unpersist strategy in ml.recommendation.ASL.train

2019-11-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29844. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26469

[jira] [Assigned] (SPARK-29844) Improper unpersist strategy in ml.recommendation.ASL.train

2019-11-12 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29844: Assignee: Dong Wang > Improper unpersist strategy in ml.recommendation.ASL.train >

[jira] [Commented] (SPARK-29493) Add MapType support for Arrow Java

2019-11-12 Thread Jalpan Randeri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972898#comment-16972898 ] Jalpan Randeri commented on SPARK-29493: Sure. i will include python changes also in my pr. >

[jira] [Updated] (SPARK-23485) Kubernetes should support node blacklist

2019-11-12 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23485: - Labels: (was: bulk-closed) > Kubernetes should support node blacklist >

[jira] [Reopened] (SPARK-23485) Kubernetes should support node blacklist

2019-11-12 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reopened SPARK-23485: -- I think this issue is still valid for current versions, it was just opened against 2.3.0, so

[jira] [Updated] (SPARK-23485) Kubernetes should support node blacklist

2019-11-12 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23485: - Affects Version/s: (was: 2.3.0) 3.0.0 2.4.0 >

[jira] [Commented] (SPARK-29493) Add MapType support for Arrow Java

2019-11-12 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972833#comment-16972833 ] Bryan Cutler commented on SPARK-29493: -- [~jalpan.randeri] this depends on SPARK-29376 for a newer

[jira] [Reopened] (SPARK-25351) Handle Pandas category type when converting from Python with Arrow

2019-11-12 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reopened SPARK-25351: -- reopening, this should be straightforward to add > Handle Pandas category type when converting

[jira] [Commented] (SPARK-29493) Add MapType support for Arrow Java

2019-11-12 Thread Jalpan Randeri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972809#comment-16972809 ] Jalpan Randeri commented on SPARK-29493: Hi Bryan,  Are you working on this issue? If not i

[jira] [Updated] (SPARK-29867) add __repr__ in Python ML Models

2019-11-12 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-29867: --- Description: In Python ML Models, some of them have ___repr, others don't. In the doctest, when

[jira] [Created] (SPARK-29867) add __repr__ in Python ML Models

2019-11-12 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-29867: -- Summary: add __repr__ in Python ML Models Key: SPARK-29867 URL: https://issues.apache.org/jira/browse/SPARK-29867 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-29762) GPU Scheduling - default task resource amount to 1

2019-11-12 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972755#comment-16972755 ] Thomas Graves commented on SPARK-29762: --- But like you mention, if we were to support something

[jira] [Resolved] (SPARK-29789) should not parse the bucket column name again when creating v2 tables

2019-11-12 Thread Ryan Blue (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Blue resolved SPARK-29789. --- Fix Version/s: 3.0.0 Resolution: Fixed > should not parse the bucket column name again when

[jira] [Commented] (SPARK-29762) GPU Scheduling - default task resource amount to 1

2019-11-12 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972750#comment-16972750 ] Thomas Graves commented on SPARK-29762: --- Ok, in the scenario above you would need 2 different

[jira] [Commented] (SPARK-29797) Read key-value metadata in Parquet files written by Apache Arrow

2019-11-12 Thread Isaac Myers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972725#comment-16972725 ] Isaac Myers commented on SPARK-29797: - To be clear, Arrow does write the metadata because it can be

[jira] [Created] (SPARK-29866) Upper case enum values

2019-11-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29866: -- Summary: Upper case enum values Key: SPARK-29866 URL: https://issues.apache.org/jira/browse/SPARK-29866 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-29865) k8s executor pods all have different prefixes in client mode

2019-11-12 Thread Marcelo Masiero Vanzin (Jira)
Marcelo Masiero Vanzin created SPARK-29865: -- Summary: k8s executor pods all have different prefixes in client mode Key: SPARK-29865 URL: https://issues.apache.org/jira/browse/SPARK-29865

[jira] [Resolved] (SPARK-29487) Ability to run Spark Kubernetes other than from /opt/spark

2019-11-12 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29487. Resolution: Duplicate This is basically allowing people to customize

[jira] [Commented] (SPARK-29762) GPU Scheduling - default task resource amount to 1

2019-11-12 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972703#comment-16972703 ] Imran Rashid commented on SPARK-29762: -- The scenario I was thinking is: application is submitted

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

2019-11-12 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972697#comment-16972697 ] Shane Knapp commented on SPARK-29106: - i would definitely be happy to have a faster VM available for

[jira] [Created] (SPARK-29864) Strict parsing of day-time strings to intervals

2019-11-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29864: -- Summary: Strict parsing of day-time strings to intervals Key: SPARK-29864 URL: https://issues.apache.org/jira/browse/SPARK-29864 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29762) GPU Scheduling - default task resource amount to 1

2019-11-12 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972661#comment-16972661 ] Thomas Graves commented on SPARK-29762: --- The code that gets the task requirements is generic and

[jira] [Created] (SPARK-29863) rename EveryAgg/AnyAgg to BoolAnd/BoolOr

2019-11-12 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29863: --- Summary: rename EveryAgg/AnyAgg to BoolAnd/BoolOr Key: SPARK-29863 URL: https://issues.apache.org/jira/browse/SPARK-29863 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-29850) sort-merge-join an empty table should not memory leak

2019-11-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29850. - Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-29762) GPU Scheduling - default task resource amount to 1

2019-11-12 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972631#comment-16972631 ] Imran Rashid commented on SPARK-29762: -- I don't really understand the complication. I know there

[jira] [Commented] (SPARK-29764) Error on Serializing POJO with java datetime property to a Parquet file

2019-11-12 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972605#comment-16972605 ] Felix Kizhakkel Jose commented on SPARK-29764: -- So it's still hard to follow the

[jira] [Updated] (SPARK-29857) [WEB UI] Support defer render the spark history summary page.

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29857: Description: When there are many applications in spark history server, the renderer of history summary

[jira] [Created] (SPARK-29862) CREATE (OR REPLACE) ... VIEW should look up catalog/table like v2 commands

2019-11-12 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-29862: -- Summary: CREATE (OR REPLACE) ... VIEW should look up catalog/table like v2 commands Key: SPARK-29862 URL: https://issues.apache.org/jira/browse/SPARK-29862 Project:

[jira] [Commented] (SPARK-29862) CREATE (OR REPLACE) ... VIEW should look up catalog/table like v2 commands

2019-11-12 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972560#comment-16972560 ] Huaxin Gao commented on SPARK-29862: I will work on this.  > CREATE (OR REPLACE) ... VIEW should

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id

[jira] [Created] (SPARK-29861) Reduce leader election downtime in Spark standalone HA

2019-11-12 Thread Robin Wolters (Jira)
Robin Wolters created SPARK-29861: - Summary: Reduce leader election downtime in Spark standalone HA Key: SPARK-29861 URL: https://issues.apache.org/jira/browse/SPARK-29861 Project: Spark

[jira] [Created] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
feiwang created SPARK-29860: --- Summary: [SQL] Fix data type mismatch issue for inSubQuery Key: SPARK-29860 URL: https://issues.apache.org/jira/browse/SPARK-29860 Project: Spark Issue Type:

[jira] [Commented] (SPARK-29859) ALTER DATABASE (SET LOCATION) should look up catalog like v2 commands

2019-11-12 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972532#comment-16972532 ] Hu Fuwang commented on SPARK-29859: --- Working on this. > ALTER DATABASE (SET LOCATION) should look up

[jira] [Commented] (SPARK-29858) ALTER DATABASE (SET DBPROPERTIES) should look up catalog like v2 commands

2019-11-12 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972530#comment-16972530 ] Hu Fuwang commented on SPARK-29858: --- Working on this. > ALTER DATABASE (SET DBPROPERTIES) should look

[jira] [Created] (SPARK-29859) ALTER DATABASE (SET LOCATION) should look up catalog like v2 commands

2019-11-12 Thread Hu Fuwang (Jira)
Hu Fuwang created SPARK-29859: - Summary: ALTER DATABASE (SET LOCATION) should look up catalog like v2 commands Key: SPARK-29859 URL: https://issues.apache.org/jira/browse/SPARK-29859 Project: Spark

[jira] [Created] (SPARK-29858) ALTER DATABASE (SET DBPROPERTIES) should look up catalog like v2 commands

2019-11-12 Thread Hu Fuwang (Jira)
Hu Fuwang created SPARK-29858: - Summary: ALTER DATABASE (SET DBPROPERTIES) should look up catalog like v2 commands Key: SPARK-29858 URL: https://issues.apache.org/jira/browse/SPARK-29858 Project: Spark

[jira] [Commented] (SPARK-20050) Kafka 0.10 DirectStream doesn't commit last processed batch's offset when graceful shutdown

2019-11-12 Thread Julian Eberius (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972524#comment-16972524 ] Julian Eberius commented on SPARK-20050: Sorry to comment on an old, closed issue, but: what is

[jira] [Commented] (SPARK-29710) Seeing offsets not resetting even when reset policy is configured explicitly

2019-11-12 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972515#comment-16972515 ] Gabor Somogyi commented on SPARK-29710: --- [~BdLearner] bump, can you provide any useful information

[jira] [Updated] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

2019-11-12 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29854: - Description: Spark Returns Empty String) {code} 0:

[jira] [Assigned] (SPARK-29855) typed literals with negative sign with proper result or exception

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29855: Assignee: Kent Yao > typed literals with negative sign with proper result or exception >

[jira] [Resolved] (SPARK-29855) typed literals with negative sign with proper result or exception

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29855. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26479

[jira] [Commented] (SPARK-29823) Improper persist strategy in mllib.clustering.KMeans.run()

2019-11-12 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972473#comment-16972473 ] Aman Omer commented on SPARK-29823: --- I will check this one. > Improper persist strategy in

[jira] [Created] (SPARK-29857) [WEB UI] Support defer render the spark history summary page.

2019-11-12 Thread feiwang (Jira)
feiwang created SPARK-29857: --- Summary: [WEB UI] Support defer render the spark history summary page. Key: SPARK-29857 URL: https://issues.apache.org/jira/browse/SPARK-29857 Project: Spark Issue

[jira] [Commented] (SPARK-29349) Support FETCH_PRIOR in Thriftserver query results fetching

2019-11-12 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972369#comment-16972369 ] Juliusz Sompolski commented on SPARK-29349: --- [~runzhiwang] it is used by Simba Spark ODBC

[jira] [Commented] (SPARK-29349) Support FETCH_PRIOR in Thriftserver query results fetching

2019-11-12 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972345#comment-16972345 ] runzhiwang commented on SPARK-29349: [~juliuszsompolski] Hi, Could you explain how to use this

[jira] [Issue Comment Deleted] (SPARK-29771) Limit executor max failures before failing the application

2019-11-12 Thread Jackey Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-29771: --- Comment: was deleted (was: This patch is mainly used in the scenario where the executor started

[jira] [Updated] (SPARK-29771) Limit executor max failures before failing the application

2019-11-12 Thread Jackey Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-29771: --- Description: ExecutorPodsAllocator does not limit the number of executor errors or deletions,

[jira] [Updated] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29854: - Description: Spark Returns Empty String) {code} 0: jdbc:hive2://10.18.19.208:23040/default>

[jira] [Commented] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972303#comment-16972303 ] Hyukjin Kwon commented on SPARK-29854: -- Other DMBSes have rpad and lpad too. Can you check? > lpad

[jira] [Commented] (SPARK-29854) lpad and rpad built in function not throw Exception for invalid len value

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972302#comment-16972302 ] Hyukjin Kwon commented on SPARK-29854: -- [~abhishek.akg] Can you please use \{code\} ... \{code\}

[jira] [Comment Edited] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread Dong Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972294#comment-16972294 ] Dong Wang edited comment on SPARK-29856 at 11/12/19 11:32 AM: -- But there is

[jira] [Commented] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread Dong Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972294#comment-16972294 ] Dong Wang commented on SPARK-29856: --- But there is only one action _collectAsMap()_ using

[jira] [Commented] (SPARK-29855) typed literals with negative sign with proper result or exception

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972292#comment-16972292 ] Hyukjin Kwon commented on SPARK-29855: -- [~Qin Yao] Please link related JIRAs for the initial

[jira] [Comment Edited] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread EnzoBnl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972224#comment-16972224 ] EnzoBnl edited comment on SPARK-29856 at 11/12/19 9:47 AM: --- If I am not wrong,

[jira] [Comment Edited] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread EnzoBnl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972224#comment-16972224 ] EnzoBnl edited comment on SPARK-29856 at 11/12/19 9:41 AM: --- Just a note: if I

[jira] [Comment Edited] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread EnzoBnl (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972224#comment-16972224 ] EnzoBnl edited comment on SPARK-29856 at 11/12/19 9:42 AM: --- Just a note: If I

[jira] [Comment Edited] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread Enzo Bonnal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972224#comment-16972224 ] Enzo Bonnal edited comment on SPARK-29856 at 11/12/19 9:39 AM: --- Just a

[jira] [Commented] (SPARK-29856) Conditional unnecessary persist on RDDs in ML algorithms

2019-11-12 Thread Enzo Bonnal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972224#comment-16972224 ] Enzo Bonnal commented on SPARK-29856: - Just a note: if I am not wrong_, findBestSplits_ may leverage

[jira] [Commented] (SPARK-29764) Error on Serializing POJO with java datetime property to a Parquet file

2019-11-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972223#comment-16972223 ] Hyukjin Kwon commented on SPARK-29764: -- priority doesn't matter about getting a help. It is just

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

2019-11-12 Thread zhao bo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972184#comment-16972184 ] zhao bo commented on SPARK-29106: - And for the above several comments I left, we hope [~shaneknapp], you

[jira] [Assigned] (SPARK-29298) Separate block manager heartbeat endpoint from driver endpoint

2019-11-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29298: --- Assignee: Lantao Jin > Separate block manager heartbeat endpoint from driver endpoint >

[jira] [Resolved] (SPARK-29298) Separate block manager heartbeat endpoint from driver endpoint

2019-11-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29298. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25971