[jira] [Updated] (SPARK-28958) pyspark.ml function parity

2019-09-04 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-28958: - Description: I looked into the hierarchy of both py and scala sides, and found that they are qu

[jira] [Created] (SPARK-28985) Pyspark ClassificationModel and RegressionModel support column setters/getters/predict

2019-09-04 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-28985: Summary: Pyspark ClassificationModel and RegressionModel support column setters/getters/predict Key: SPARK-28985 URL: https://issues.apache.org/jira/browse/SPARK-28985

[jira] [Updated] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread cheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cheng updated SPARK-28984: -- Attachment: multi_csv2.csv > csv files should support multi-character separators > ---

[jira] [Updated] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28984: Shepherd: (was: Yuming Wang) > csv files should support multi-character separators > ---

[jira] [Updated] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28984: Component/s: (was: Input/Output) SQL > csv files should support multi-charact

[jira] [Updated] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28984: Target Version/s: (was: 2.3.2) > csv files should support multi-character separators > -

[jira] [Updated] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28984: Affects Version/s: (was: 2.3.2) 3.0.0 > csv files should support multi-

[jira] [Created] (SPARK-28984) csv files should support multi-character separators

2019-09-04 Thread cheng (Jira)
cheng created SPARK-28984: - Summary: csv files should support multi-character separators Key: SPARK-28984 URL: https://issues.apache.org/jira/browse/SPARK-28984 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-28976) Use KeyLock to simplify MapOutputTracker.getStatuses

2019-09-04 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-28976. -- Fix Version/s: 3.0.0 Resolution: Fixed > Use KeyLock to simplify MapOutputTracker.getSt

[jira] [Resolved] (SPARK-28830) Document UNCACHE TABLE in SQL Reference

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28830. - Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed > Document UNCACHE TABLE in

[jira] [Commented] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-04 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923041#comment-16923041 ] Huaxin Gao commented on SPARK-28969: Thanks [~podongfeng] I will work on this. > O

[jira] [Updated] (SPARK-28983) Can not pruning hive partitions by Or predicate

2019-09-04 Thread huajian lan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huajian lan updated SPARK-28983: Description: Assume the hive partition table has p1/p2 partition column and c1 non-partition colu

[jira] [Updated] (SPARK-28983) Can not pruning hive partitions by Or predicate

2019-09-04 Thread huajian lan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huajian lan updated SPARK-28983: Description: Assume the hive partition table has p1/p2 partition column and c1 non-partition colu

[jira] [Created] (SPARK-28983) Can not pruning hive partitions by Or predicate

2019-09-04 Thread huajian lan (Jira)
huajian lan created SPARK-28983: --- Summary: Can not pruning hive partitions by Or predicate Key: SPARK-28983 URL: https://issues.apache.org/jira/browse/SPARK-28983 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-28979) DataSourceV2: Rename UnresolvedTable

2019-09-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28979. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25683 [https://gith

[jira] [Assigned] (SPARK-28979) DataSourceV2: Rename UnresolvedTable

2019-09-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28979: --- Assignee: Ryan Blue > DataSourceV2: Rename UnresolvedTable > --

[jira] [Commented] (SPARK-28934) Add `spark.sql.compatiblity.mode`

2019-09-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923026#comment-16923026 ] Dongjoon Hyun commented on SPARK-28934: --- Thanks, Gengliang! > Add `spark.sql.comp

[jira] [Updated] (SPARK-28982) Support ThriftServer GetTypeInfoOperation for Spark's own type

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28982: Parent: SPARK-28426 Issue Type: Sub-task (was: Improvement) > Support ThriftServer GetTyp

[jira] [Updated] (SPARK-28982) Support ThriftServer GetTypeInfoOperation for Spark's own type

2019-09-04 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28982: Affects Version/s: (was: 2.4.0) > Support ThriftServer GetTypeInfoOperation for Spark's own ty

[jira] [Created] (SPARK-28982) Support ThriftServer GetTypeInfoOperation for Spark's own type

2019-09-04 Thread angerszhu (Jira)
angerszhu created SPARK-28982: - Summary: Support ThriftServer GetTypeInfoOperation for Spark's own type Key: SPARK-28982 URL: https://issues.apache.org/jira/browse/SPARK-28982 Project: Spark Iss

[jira] [Reopened] (SPARK-19147) netty throw NPE

2019-09-04 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reopened SPARK-19147: - > netty throw NPE > --- > > Key: SPARK-19147 > URL: http

[jira] [Commented] (SPARK-19147) netty throw NPE

2019-09-04 Thread Saisai Shao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923014#comment-16923014 ] Saisai Shao commented on SPARK-19147: - Yes, we also met the similar issue when execu

[jira] [Updated] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-04 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-28969: - Parent: SPARK-28958 Issue Type: Sub-task (was: Improvement) > OneVsRestModel in the py

[jira] [Commented] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-04 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922994#comment-16922994 ] zhengruifeng commented on SPARK-28969: -- friendly ping [~huaxingao] > OneVsRestMode

[jira] [Commented] (SPARK-28516) Data Type Formatting Functions: `to_char`

2019-09-04 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922969#comment-16922969 ] YoungGyu Chun commented on SPARK-28516: --- I'm working on this! > Data Type Formatt

[jira] [Resolved] (SPARK-28907) Review invalid usage of new Configuration()

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28907. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25616 [https://github.c

[jira] [Assigned] (SPARK-28907) Review invalid usage of new Configuration()

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28907: - Assignee: Xianjin YE > Review invalid usage of new Configuration() > --

[jira] [Created] (SPARK-28981) Missing library for reading/writing Snappy-compressed files

2019-09-04 Thread Paul Schweigert (Jira)
Paul Schweigert created SPARK-28981: --- Summary: Missing library for reading/writing Snappy-compressed files Key: SPARK-28981 URL: https://issues.apache.org/jira/browse/SPARK-28981 Project: Spark

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922936#comment-16922936 ] Jungtaek Lim commented on SPARK-28770: -- Great! Thanks both of you for understanding

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2019-09-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922889#comment-16922889 ] Dongjoon Hyun commented on SPARK-28900: --- Take care, Shane! > Test Pyspark, SparkR

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Wing Yew Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922925#comment-16922925 ] Wing Yew Poon commented on SPARK-28770: --- As [~irashid] noted, following an offline

[jira] [Updated] (SPARK-28980) Remove most remaining deprecated items since <= 2.2.0 for 3.0

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28980: -- Component/s: MLlib Summary: Remove most remaining deprecated items since <= 2.2.0 for 3.0 (wa

[jira] [Updated] (SPARK-28977) JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28977: -- Target Version/s: (was: 2.4.3) > JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page > ---

[jira] [Updated] (SPARK-28977) JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28977: -- Fix Version/s: (was: 2.4.3) Priority: Minor (was: Major) > JDBC Dataframe Reader Doc Doe

[jira] [Created] (SPARK-28980) Remove remaining deprecated items since <= 2.2.0 for 3.0

2019-09-04 Thread Sean Owen (Jira)
Sean Owen created SPARK-28980: - Summary: Remove remaining deprecated items since <= 2.2.0 for 3.0 Key: SPARK-28980 URL: https://issues.apache.org/jira/browse/SPARK-28980 Project: Spark Issue Type

[jira] [Created] (SPARK-28979) DataSourceV2: Rename UnresolvedTable

2019-09-04 Thread Ryan Blue (Jira)
Ryan Blue created SPARK-28979: - Summary: DataSourceV2: Rename UnresolvedTable Key: SPARK-28979 URL: https://issues.apache.org/jira/browse/SPARK-28979 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-28978) PySpark: Can't pass more than 256 arguments to a UDF

2019-09-04 Thread Jim Fulton (Jira)
Jim Fulton created SPARK-28978: -- Summary: PySpark: Can't pass more than 256 arguments to a UDF Key: SPARK-28978 URL: https://issues.apache.org/jira/browse/SPARK-28978 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-11150) Dynamic partition pruning

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-11150. - Fix Version/s: 3.0.0 Assignee: Maryann Xue Resolution: Fixed > Dynamic partition pruning

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922825#comment-16922825 ] Imran Rashid commented on SPARK-28770: -- Wing yew and I chatted about this offline a

[jira] [Created] (SPARK-28977) JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page

2019-09-04 Thread Christopher Hoshino-Fish (Jira)
Christopher Hoshino-Fish created SPARK-28977: Summary: JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page Key: SPARK-28977 URL: https://issues.apache.org/jira/browse/SPARK-28977

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922777#comment-16922777 ] Jungtaek Lim commented on SPARK-28770: -- As I commented in PR, I'm not sure EventMon

[jira] [Comment Edited] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922777#comment-16922777 ] Jungtaek Lim edited comment on SPARK-28770 at 9/4/19 7:21 PM:

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-04 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922771#comment-16922771 ] Imran Rashid commented on SPARK-28770: -- I'm not sure I follow ... the metrics from

[jira] [Resolved] (SPARK-28811) Document SHOW TBLPROPERTIES in SQL Reference.

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28811. - Fix Version/s: 3.0.0 Assignee: Dilip Biswal Resolution: Fixed > Document SHOW TBLPROPERT

[jira] [Resolved] (SPARK-28808) Document SHOW FUNCTIONS in SQL Reference.

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28808. - Fix Version/s: 3.0.0 Resolution: Fixed > Document SHOW FUNCTIONS in SQL Reference. >

[jira] [Assigned] (SPARK-28808) Document SHOW FUNCTIONS in SQL Reference.

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-28808: --- Assignee: Dilip Biswal > Document SHOW FUNCTIONS in SQL Reference. > --

[jira] [Assigned] (SPARK-28976) Use KeyLock to simplify MapOutputTracker.getStatuses

2019-09-04 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-28976: Assignee: Shixiong Zhu > Use KeyLock to simplify MapOutputTracker.getStatuses > -

[jira] [Created] (SPARK-28976) Use KeyLock to simplify MapOutputTracker.getStatuses

2019-09-04 Thread Shixiong Zhu (Jira)
Shixiong Zhu created SPARK-28976: Summary: Use KeyLock to simplify MapOutputTracker.getStatuses Key: SPARK-28976 URL: https://issues.apache.org/jira/browse/SPARK-28976 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-25151) Apply Apache Commons Pool to KafkaDataConsumer

2019-09-04 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25151: -- Assignee: Jungtaek Lim > Apply Apache Commons Pool to KafkaDataConsumer > ---

[jira] [Resolved] (SPARK-25151) Apply Apache Commons Pool to KafkaDataConsumer

2019-09-04 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25151. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 22138 [https:

[jira] [Updated] (SPARK-28975) How do I overwrite a piece of data and recalculate it?

2019-09-04 Thread ruiliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruiliang updated SPARK-28975: - Attachment: image-2019-09-05-01-14-26-620.png > How do I overwrite a piece of data and recalculate it? >

[jira] [Updated] (SPARK-28975) How do I overwrite a piece of data and recalculate it?

2019-09-04 Thread ruiliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruiliang updated SPARK-28975: - Attachment: (was: QQ图片20190905011256.png) > How do I overwrite a piece of data and recalculate it? >

[jira] [Updated] (SPARK-28975) How do I overwrite a piece of data and recalculate it?

2019-09-04 Thread ruiliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruiliang updated SPARK-28975: - Description:  翻译: I have a requirement to make real-time statistics of the total amount and quantity o

[jira] [Updated] (SPARK-28975) How do I overwrite a piece of data and recalculate it?

2019-09-04 Thread ruiliang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruiliang updated SPARK-28975: - Attachment: QQ图片20190905011256.png > How do I overwrite a piece of data and recalculate it? > --

[jira] [Commented] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-04 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922684#comment-16922684 ] shane knapp commented on SPARK-28953: - [~skonto] we're currently on minikube v0.34.1

[jira] [Created] (SPARK-28975) How do I overwrite a piece of data and recalculate it?

2019-09-04 Thread ruiliang (Jira)
ruiliang created SPARK-28975: Summary: How do I overwrite a piece of data and recalculate it? Key: SPARK-28975 URL: https://issues.apache.org/jira/browse/SPARK-28975 Project: Spark Issue Type: Qu

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2019-09-04 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922681#comment-16922681 ] shane knapp commented on SPARK-28900: - hey everyone, i have to back-burner this issu

[jira] [Resolved] (SPARK-28931) Fix couple of bugs in FsHistoryProviderSuite

2019-09-04 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-28931. Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed > Fix couple

[jira] [Resolved] (SPARK-28901) SparkThriftServer SparkExecuteStatementOpration handle cancel status.

2019-09-04 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28901. - Fix Version/s: 3.0.0 Assignee: angerszhu Resolution: Fixed > SparkThriftServer SparkExec

[jira] [Assigned] (SPARK-27492) GPU scheduling - High level user documentation

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-27492: - Assignee: Thomas Graves > GPU scheduling - High level user documentation >

[jira] [Comment Edited] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-04 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922547#comment-16922547 ] Stavros Kontopoulos edited comment on SPARK-28953 at 9/4/19 3:43 PM: -

[jira] [Commented] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-04 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922547#comment-16922547 ] Stavros Kontopoulos commented on SPARK-28953: - [~srowen] thanks I will have

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Resolved] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-28577. --- Fix Version/s: 3.0.0 Assignee: Yang Jie Resolution: Fixed > Ensure executorM

[jira] [Commented] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-04 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922480#comment-16922480 ] Sean Owen commented on SPARK-28953: --- [~holdenk] Looks like this is the same issue as a

[jira] [Created] (SPARK-28974) centralize the Data Source V2 table capability checks

2019-09-04 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-28974: --- Summary: centralize the Data Source V2 table capability checks Key: SPARK-28974 URL: https://issues.apache.org/jira/browse/SPARK-28974 Project: Spark Issue Typ

[jira] [Updated] (SPARK-28973) Add TimeType to Catalyst

2019-09-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-28973: --- Description: The time type should represent local time in microsecond precision with valid range of

[jira] [Updated] (SPARK-28973) Add TimeType to Catalyst

2019-09-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-28973: --- Description: The time type should represent local time in microsecond precision with valid range of

[jira] [Created] (SPARK-28973) Add TimeType to Catalyst

2019-09-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-28973: -- Summary: Add TimeType to Catalyst Key: SPARK-28973 URL: https://issues.apache.org/jira/browse/SPARK-28973 Project: Spark Issue Type: Sub-task Component

[jira] [Commented] (SPARK-28972) [Spark] spark.memory.offHeap.size description require to update in document

2019-09-04 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922414#comment-16922414 ] pavithra ramachandran commented on SPARK-28972: --- i will work on this > [S

[jira] [Created] (SPARK-28972) [Spark] spark.memory.offHeap.size description require to update in document

2019-09-04 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-28972: Summary: [Spark] spark.memory.offHeap.size description require to update in document Key: SPARK-28972 URL: https://issues.apache.org/jira/browse/SPARK-28972

[jira] [Closed] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi closed SPARK-28641. - > MicroBatchExecution committed offsets greater than available offsets > ---

[jira] [Resolved] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-04 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-28641. --- Resolution: Duplicate > MicroBatchExecution committed offsets greater than available offsets

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

2019-09-04 Thread Vadim Panov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922382#comment-16922382 ] Vadim Panov commented on SPARK-13446: - Did some research on the issue: * The offend

[jira] [Commented] (SPARK-28274) Convert and port 'pgSQL/window.sql' into UDF test base

2019-09-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922357#comment-16922357 ] Hyukjin Kwon commented on SPARK-28274: -- Let me actually get rid of this test here f

[jira] [Resolved] (SPARK-28274) Convert and port 'pgSQL/window.sql' into UDF test base

2019-09-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28274. -- Resolution: Later > Convert and port 'pgSQL/window.sql' into UDF test base > -

[jira] [Created] (SPARK-28971) Convert and port 'pgSQL/aggregates_part4.sql' into UDF test base

2019-09-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-28971: Summary: Convert and port 'pgSQL/aggregates_part4.sql' into UDF test base Key: SPARK-28971 URL: https://issues.apache.org/jira/browse/SPARK-28971 Project: Spark

[jira] [Commented] (SPARK-28934) Add `spark.sql.compatiblity.mode`

2019-09-04 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922295#comment-16922295 ] Gengliang Wang commented on SPARK-28934: Anyone working on it? If not, I will ta

[jira] [Commented] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-04 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922292#comment-16922292 ] Stavros Kontopoulos commented on SPARK-28953: - [~shaneknapp] I can attach th

[jira] [Commented] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException

2019-09-04 Thread Colin Ma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922207#comment-16922207 ] Colin Ma commented on SPARK-28340: -- In general, the exception will be caught and proces