[jira] [Created] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-32379: --- Summary: docker based spark release script should use correct CRAN repo. Key: SPARK-32379 URL: https://issues.apache.org/jira/browse/SPARK-32379 Project: Spark

[jira] [Assigned] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32379: Assignee: Apache Spark > docker based spark release script should use correct CRAN repo.

[jira] [Commented] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161846#comment-17161846 ] Apache Spark commented on SPARK-32379: -- User 'ScrapCodes' has created a pull reques

[jira] [Assigned] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32379: Assignee: (was: Apache Spark) > docker based spark release script should use correct

[jira] [Commented] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161848#comment-17161848 ] Apache Spark commented on SPARK-32379: -- User 'ScrapCodes' has created a pull reques

[jira] [Resolved] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32379. -- Fix Version/s: 2.4.7 Resolution: Fixed Issue resolved by pull request 29177 [https://gi

[jira] [Assigned] (SPARK-32379) docker based spark release script should use correct CRAN repo.

2020-07-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32379: Assignee: Prashant Sharma > docker based spark release script should use correct CRAN rep

[jira] [Created] (SPARK-32380) sparksql cannot access hbase external table in hive

2020-07-21 Thread deyzhong (Jira)
deyzhong created SPARK-32380: Summary: sparksql cannot access hbase external table in hive Key: SPARK-32380 URL: https://issues.apache.org/jira/browse/SPARK-32380 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

2020-07-21 Thread Krish (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161952#comment-17161952 ] Krish commented on SPARK-32317: --- Yes I do agree with your second point, if we map required

[jira] [Updated] (SPARK-32380) sparksql cannot access hbase external table in hive

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table * {code:java} create 'hbase_test2', 'cf1' {code} * create

[jira] [Updated] (SPARK-32380) sparksql cannot access hbase external table in hive

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table * {code:java} create 'hbase_test2', 'cf1' {code} * create

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data on hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Summary: sparksql cannot access hive table while data on hbase (was: sparksql cannot access hbase exter

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Summary: sparksql cannot access hive table while data in hbase (was: sparksql cannot access hive table

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table * {code:java} create 'hbase_test2', 'cf1' {code} * create

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table * {code:java} create 'hbase_test2', 'cf1' {code} * create

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table {code:java} create 'hbase_test2', 'cf1'{code} * create hive

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table {code:java} hbase(main):001:0>create 'hbase_test1', 'cf1' hb

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161959#comment-17161959 ] deyzhong commented on SPARK-32380: -- I have solved this bug by modified TableReader.scal

[jira] [Updated] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread deyzhong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deyzhong updated SPARK-32380: - Description: * step1: create hbase table {code:java} hbase(main):001:0>create 'hbase_test1', 'cf1' hb

[jira] [Resolved] (SPARK-32363) Flaky pip installation test in Jenkins

2020-07-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32363. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29117 [https://gi

[jira] [Assigned] (SPARK-32363) Flaky pip installation test in Jenkins

2020-07-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32363: Assignee: Hyukjin Kwon > Flaky pip installation test in Jenkins > ---

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162058#comment-17162058 ] Apache Spark commented on SPARK-32380: -- User 'DeyinZhong' has created a pull reques

[jira] [Assigned] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32380: Assignee: (was: Apache Spark) > sparksql cannot access hive table while data in hbase

[jira] [Commented] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162059#comment-17162059 ] Apache Spark commented on SPARK-32380: -- User 'DeyinZhong' has created a pull reques

[jira] [Assigned] (SPARK-32380) sparksql cannot access hive table while data in hbase

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32380: Assignee: Apache Spark > sparksql cannot access hive table while data in hbase >

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-07-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162063#comment-17162063 ] Xinli Shang commented on SPARK-26345: - [~yumwang][~FelixKJose], you can assign this

[jira] [Updated] (SPARK-32377) CaseInsensitiveMap should be deterministic for addition

2020-07-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32377: -- Fix Version/s: 2.4.7 > CaseInsensitiveMap should be deterministic for addition > -

[jira] [Commented] (SPARK-32334) Investigate commonizing Columnar and Row data transformations

2020-07-21 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162147#comment-17162147 ] Robert Joseph Evans commented on SPARK-32334: - I think I can get the convers

[jira] [Commented] (SPARK-32348) Get tests working for Scala 2.13 build

2020-07-21 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162280#comment-17162280 ] Sean R. Owen commented on SPARK-32348: -- I've found a few more easy test fixes, but

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-07-21 Thread Felix Kizhakkel Jose (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162311#comment-17162311 ] Felix Kizhakkel Jose commented on SPARK-26345: -- [~sha...@uber.com] I don't

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-07-21 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162313#comment-17162313 ] Holden Karau commented on SPARK-26345: -- We don't assign issues normally until after

[jira] [Commented] (SPARK-32381) Expose the ability for users to use parallel file & avoid location information discovery in RDDs

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162314#comment-17162314 ] Apache Spark commented on SPARK-32381: -- User 'holdenk' has created a pull request f

[jira] [Assigned] (SPARK-32381) Expose the ability for users to use parallel file & avoid location information discovery in RDDs

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32381: Assignee: Apache Spark > Expose the ability for users to use parallel file & avoid locati

[jira] [Assigned] (SPARK-32381) Expose the ability for users to use parallel file & avoid location information discovery in RDDs

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32381: Assignee: (was: Apache Spark) > Expose the ability for users to use parallel file & a

[jira] [Created] (SPARK-32381) Expose the ability for users to use parallel file & avoid location information discovery in RDDs

2020-07-21 Thread Holden Karau (Jira)
Holden Karau created SPARK-32381: Summary: Expose the ability for users to use parallel file & avoid location information discovery in RDDs Key: SPARK-32381 URL: https://issues.apache.org/jira/browse/SPARK-32381

[jira] [Created] (SPARK-32382) Override table renaming in JDBC dialects

2020-07-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32382: -- Summary: Override table renaming in JDBC dialects Key: SPARK-32382 URL: https://issues.apache.org/jira/browse/SPARK-32382 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-17333) Make pyspark interface friendly with mypy static analysis

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17333: Assignee: (was: Apache Spark) > Make pyspark interface friendly with mypy static anal

[jira] [Commented] (SPARK-17333) Make pyspark interface friendly with mypy static analysis

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162339#comment-17162339 ] Apache Spark commented on SPARK-17333: -- User 'Fokko' has created a pull request for

[jira] [Assigned] (SPARK-17333) Make pyspark interface friendly with mypy static analysis

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17333: Assignee: Apache Spark > Make pyspark interface friendly with mypy static analysis >

[jira] [Updated] (SPARK-32377) CaseInsensitiveMap should be deterministic for addition

2020-07-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32377: -- Reporter: Girish A Pandit (was: Dongjoon Hyun) > CaseInsensitiveMap should be deterministic f

[jira] [Resolved] (SPARK-24266) Spark client terminates while driver is still running

2020-07-21 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-24266. -- Fix Version/s: 3.1.0 Target Version/s: 3.1.0 (was: 2.4.7, 3.1.0) Resolution:

[jira] [Resolved] (SPARK-32286) Coalesce bucketed tables for shuffled hash join if applicable

2020-07-21 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32286. -- Fix Version/s: 3.1.0 Assignee: Cheng Su Resolution: Fixed Resolved by 

[jira] [Created] (SPARK-32383) Preserve hash join (BHJ and SHJ) stream side ordering

2020-07-21 Thread Cheng Su (Jira)
Cheng Su created SPARK-32383: Summary: Preserve hash join (BHJ and SHJ) stream side ordering Key: SPARK-32383 URL: https://issues.apache.org/jira/browse/SPARK-32383 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-32383) Preserve hash join (BHJ and SHJ) stream side ordering

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162413#comment-17162413 ] Apache Spark commented on SPARK-32383: -- User 'c21' has created a pull request for t

[jira] [Assigned] (SPARK-32383) Preserve hash join (BHJ and SHJ) stream side ordering

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32383: Assignee: Apache Spark > Preserve hash join (BHJ and SHJ) stream side ordering >

[jira] [Assigned] (SPARK-32383) Preserve hash join (BHJ and SHJ) stream side ordering

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32383: Assignee: (was: Apache Spark) > Preserve hash join (BHJ and SHJ) stream side ordering

[jira] [Commented] (SPARK-23844) Socket Stream recovering from checkpoint will throw exception

2020-07-21 Thread pengzhiwei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162438#comment-17162438 ] pengzhiwei commented on SPARK-23844: Thanks [~jerryshao2015],I have meet the same is

[jira] [Updated] (SPARK-32330) Preserve shuffled hash join build side partitioning

2020-07-21 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-32330: Priority: Major (was: Trivial) > Preserve shuffled hash join build side partitioning > --

[jira] [Assigned] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-07-21 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-32350: Assignee: Baohe Zhang > Add batch write support on LevelDB to improve performance of Hybr

[jira] [Resolved] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-07-21 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-32350. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29149 [https://gi

[jira] [Updated] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-07-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32059: -- Affects Version/s: (was: 3.0.0) 3.1.0 > Nested Schema Pruning not W

[jira] [Commented] (SPARK-32351) Partially pushed partition filters are not explained

2020-07-21 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162477#comment-17162477 ] pavithra ramachandran commented on SPARK-32351: --- i would like to check thi

[jira] [Commented] (SPARK-32003) Shuffle files for lost executor are not unregistered if fetch failure occurs after executor is lost

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162479#comment-17162479 ] Apache Spark commented on SPARK-32003: -- User 'wypoon' has created a pull request fo

[jira] [Resolved] (SPARK-31922) "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

2020-07-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31922. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28746 [https://

[jira] [Assigned] (SPARK-31922) "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

2020-07-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31922: - Assignee: wuyi > "RpcEnv already stopped" error when exit spark-shell with local-cluste

[jira] [Commented] (SPARK-21117) Built-in SQL Function Support - WIDTH_BUCKET

2020-07-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162520#comment-17162520 ] Apache Spark commented on SPARK-21117: -- User 'maropu' has created a pull request fo