[jira] [Created] (SPARK-31344) Polish implementation of barrier() and allGather()

2020-04-03 Thread wuyi (Jira)
wuyi created SPARK-31344: Summary: Polish implementation of barrier() and allGather() Key: SPARK-31344 URL: https://issues.apache.org/jira/browse/SPARK-31344 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-31231) Support setuptools 46.1.0+ in PySpark packaging

2020-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074961#comment-17074961 ] Hyukjin Kwon commented on SPARK-31231: -- This was fixed in setuptools https://github

[jira] [Resolved] (SPARK-31231) Support setuptools 46.1.0+ in PySpark packaging

2020-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31231. -- Assignee: Hyukjin Kwon Resolution: Fixed > Support setuptools 46.1.0+ in PySpark packagi

[jira] [Commented] (SPARK-31308) Make Python dependencies available for Non-PySpark applications

2020-04-03 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074889#comment-17074889 ] L. C. Hsieh commented on SPARK-31308: - Thank you [~dongjoon] > Make Python dependen

[jira] [Created] (SPARK-31343) Check codegen does not fail on expressions with special characters in string parameters

2020-04-03 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31343: -- Summary: Check codegen does not fail on expressions with special characters in string parameters Key: SPARK-31343 URL: https://issues.apache.org/jira/browse/SPARK-31343 P

[jira] [Created] (SPARK-31342) Fail by default if Parquet DATE or TIMESTAMP data is before October 15, 1582

2020-04-03 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-31342: - Summary: Fail by default if Parquet DATE or TIMESTAMP data is before October 15, 1582 Key: SPARK-31342 URL: https://issues.apache.org/jira/browse/SPARK-31342 Projec

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-04-03 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074745#comment-17074745 ] Bruce Robbins commented on SPARK-30951: --- {quote} we can fail by default when readi

[jira] [Created] (SPARK-31341) Spark documentation incorrectly claims 3.8 compatibility

2020-04-03 Thread Daniel King (Jira)
Daniel King created SPARK-31341: --- Summary: Spark documentation incorrectly claims 3.8 compatibility Key: SPARK-31341 URL: https://issues.apache.org/jira/browse/SPARK-31341 Project: Spark Issue

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074689#comment-17074689 ] Dongjoon Hyun commented on SPARK-25102: --- Are we going to have 2.4.7 or 2.4.8? For

[jira] [Comment Edited] (SPARK-31276) Contrived working example that works with multiple URI file storages for Spark cluster mode

2020-04-03 Thread Jim Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074677#comment-17074677 ] Jim Huang edited comment on SPARK-31276 at 4/3/20, 3:51 PM:

[jira] [Reopened] (SPARK-31276) Contrived working example that works with multiple URI file storages for Spark cluster mode

2020-04-03 Thread Jim Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Huang reopened SPARK-31276: --- Clarify the question with more succinct scenario that describe the the challenge of URI not being able

[jira] [Commented] (SPARK-31276) Contrived working example that works with multiple URI file storages for Spark cluster mode

2020-04-03 Thread Jim Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074676#comment-17074676 ] Jim Huang commented on SPARK-31276: --- The fault is my for not framing the scenario more

[jira] [Updated] (SPARK-31340) No call to destroy() for filter in SparkHistory

2020-04-03 Thread thierry accart (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thierry accart updated SPARK-31340: --- Summary: No call to destroy() for filter in SparkHistory (was: No call to destroy() for aut

[jira] [Created] (SPARK-31340) No call to destroy() for authentication filter in SparkHistory

2020-04-03 Thread thierry accart (Jira)
thierry accart created SPARK-31340: -- Summary: No call to destroy() for authentication filter in SparkHistory Key: SPARK-31340 URL: https://issues.apache.org/jira/browse/SPARK-31340 Project: Spark

[jira] [Resolved] (SPARK-31327) write spark version to avro file metadata

2020-04-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31327. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28102 [https://gith

[jira] [Updated] (SPARK-30840) Add version property for ConfigEntry and ConfigBuilder

2020-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30840: - Fix Version/s: 3.0.0 > Add version property for ConfigEntry and ConfigBuilder >

[jira] [Updated] (SPARK-31339) Changed PipelineModel(...) to self.cls(...) in pyspark.ml.pipeline.PipelineModelReader.load()

2020-04-03 Thread Suraj (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suraj updated SPARK-31339: -- Description: PR: [https://github.com/apache/spark/pull/28110] What changes were proposed in this pull request

[jira] [Updated] (SPARK-31339) Changed PipelineModel(...) to self.cls(...) in pyspark.ml.pipeline.PipelineModelReader.load()

2020-04-03 Thread Suraj (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suraj updated SPARK-31339: -- Description: PR: [https://github.com/apache/spark/pull/28110] * What changes were proposed in this pull reque

[jira] [Updated] (SPARK-31339) Changed PipelineModel(...) to self.cls(...) in pyspark.ml.pipeline.PipelineModelReader.load()

2020-04-03 Thread Suraj (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suraj updated SPARK-31339: -- Description: PR: [https://github.com/apache/spark/pull/28110] What changes were proposed in this pull request

[jira] [Created] (SPARK-31339) Changed PipelineModel(...) to self.cls(...) in pyspark.ml.pipeline.PipelineModelReader.load()

2020-04-03 Thread Suraj (Jira)
Suraj created SPARK-31339: - Summary: Changed PipelineModel(...) to self.cls(...) in pyspark.ml.pipeline.PipelineModelReader.load() Key: SPARK-31339 URL: https://issues.apache.org/jira/browse/SPARK-31339 Proje

[jira] [Updated] (SPARK-31338) Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key.

2020-04-03 Thread Mohit Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Dave updated SPARK-31338: --- Description: h2. *Our Use-case Details:* While reading from a jdbc source using spark sql, we are u

[jira] [Updated] (SPARK-31338) Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key.

2020-04-03 Thread Mohit Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Dave updated SPARK-31338: --- Description: *Our Use-case Details:* While reading from a jdbc source using spark sql, we are using

[jira] [Updated] (SPARK-31338) Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key.

2020-04-03 Thread Mohit Dave (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Dave updated SPARK-31338: --- Description: *Our Use-case Details:* While reading from a jdbc source using spark sql, we are using

[jira] [Created] (SPARK-31338) Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key.

2020-04-03 Thread Mohit Dave (Jira)
Mohit Dave created SPARK-31338: -- Summary: Spark SQL JDBC Data Source partitioned read : Spark SQL does not honor for NOT NULL table definition of partition key. Key: SPARK-31338 URL: https://issues.apache.org/jira/br

[jira] [Commented] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2020-04-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074434#comment-17074434 ] Wenchen Fan commented on SPARK-25102: - I'd like to propose to backport it to 2.4. It

[jira] [Commented] (SPARK-31272) Support DB2 Kerberos login in JDBC connector

2020-04-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074421#comment-17074421 ] Gabor Somogyi commented on SPARK-31272: --- The code is ready but it's depending on t

[jira] [Created] (SPARK-31336) Support Oracle Kerberos login in JDBC connector

2020-04-03 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31336: - Summary: Support Oracle Kerberos login in JDBC connector Key: SPARK-31336 URL: https://issues.apache.org/jira/browse/SPARK-31336 Project: Spark Issue Type:

[jira] [Created] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-04-03 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31337: - Summary: Support MS Sql Kerberos login in JDBC connector Key: SPARK-31337 URL: https://issues.apache.org/jira/browse/SPARK-31337 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

2020-04-03 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074390#comment-17074390 ] Ismaël Mejía commented on SPARK-31330: -- Ah thanks for letting me know about the mai

[jira] [Issue Comment Deleted] (SPARK-31334) Use agg column in Having clause behave different with column type

2020-04-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-31334: -- Comment: was deleted (was: cc [~cloud_fan] [~yumwang] ) > Use agg column in Having clause behave diff

[jira] [Commented] (SPARK-31334) Use agg column in Having clause behave different with column type

2020-04-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074381#comment-17074381 ] angerszhu commented on SPARK-31334: --- I have found the reason, In analyzer when  logica

[jira] [Commented] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

2020-04-03 Thread philipse (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074361#comment-17074361 ] philipse commented on SPARK-18681: -- [~michael] any news for this issue ? i meet the sam

[jira] [Created] (SPARK-31335) Add try function support

2020-04-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-31335: Summary: Add try function support Key: SPARK-31335 URL: https://issues.apache.org/jira/browse/SPARK-31335 Project: Spark Issue Type: New Feature Components

[jira] [Resolved] (SPARK-31249) Flaky Test: CoarseGrainedSchedulerBackendSuite.custom log url for Spark UI is applied

2020-04-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31249. -- Fix Version/s: 3.0.0 Assignee: wuyi Resolution: Fixed Fixed in https://github.