[jira] [Updated] (SPARK-36283) Bug when creating dataframe without schema and with Arrow disabled

2021-07-25 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-36283: --- Description: A reproducible small repo can be found here: https://github.com/darcy-shen/spark-36283

[jira] [Updated] (SPARK-36283) Bug when creating dataframe without schema and with Arrow disabled

2021-07-25 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-36283: --- Description: h2. Case 1: Create PySpark Dataframe using Pandas DataFrame with Arrow disabled and w

[jira] [Updated] (SPARK-36283) Bug when creating dataframe without schema and with Arrow disabled

2021-07-25 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-36283: --- Description: h2. Case 1 Create PySpark Dataframe using Pandas DataFrame with Arrow disabled and w

[jira] [Updated] (SPARK-36283) Bug when creating dataframe without schema and with Arrow disabled

2021-07-25 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darcy Shen updated SPARK-36283: --- Description: h2. Case 1 {code:python} spark = SparkSession.builder.getOrCreate() spark.conf.set("spa

[jira] [Commented] (SPARK-36285) Skip MiMa in PySpark GHA job

2021-07-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387073#comment-17387073 ] Dongjoon Hyun commented on SPARK-36285: --- Like SPARK-36198, could you make a PR for

[jira] [Created] (SPARK-36285) Skip MiMa in PySpark GHA job

2021-07-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-36285: - Summary: Skip MiMa in PySpark GHA job Key: SPARK-36285 URL: https://issues.apache.org/jira/browse/SPARK-36285 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36282. -- Resolution: Duplicate > Spark ThriftServer does not search hive external table which storage i

[jira] [Created] (SPARK-36284) Add shuffle checksum support for push-based shuffle

2021-07-25 Thread wuyi (Jira)
wuyi created SPARK-36284: Summary: Add shuffle checksum support for push-based shuffle Key: SPARK-36284 URL: https://issues.apache.org/jira/browse/SPARK-36284 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387025#comment-17387025 ] Apache Spark commented on SPARK-36242: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387023#comment-17387023 ] Apache Spark commented on SPARK-36242: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-36242) Ensure spill file closed before set success to true in ExternalSorter.spillMemoryIteratorToDisk method

2021-07-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387022#comment-17387022 ] Apache Spark commented on SPARK-36242: -- User 'LuciferYang' has created a pull reque

[jira] [Created] (SPARK-36283) Bug when creating dataframe without schema and with Arrow disabled

2021-07-25 Thread Darcy Shen (Jira)
Darcy Shen created SPARK-36283: -- Summary: Bug when creating dataframe without schema and with Arrow disabled Key: SPARK-36283 URL: https://issues.apache.org/jira/browse/SPARK-36283 Project: Spark

[jira] [Updated] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Yu Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-36282: Description: hive version is 2.3.3, hbase version is 2.4.2; CREATE EXTERNAL TABLE `temp.mapping_from_hbas

[jira] [Updated] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Yu Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-36282: Affects Version/s: (was: 3.0.0) 3.0.1 > Spark ThriftServer does not search hive

[jira] [Updated] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Yu Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-36282: Description: hive version is 2.3.3, hbase version is 2.4.2; CREATE EXTERNAL TABLE `temp.mapping_from_hbas

[jira] [Updated] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Yu Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang updated SPARK-36282: Attachment: screenshot-1.png > Spark ThriftServer does not search hive external table which storage in hba

[jira] [Commented] (SPARK-31809) Infer IsNotNull for non null intolerant child of null intolerant in join condition

2021-07-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386991#comment-17386991 ] Yuming Wang commented on SPARK-31809: - {code:java} spark.sql("create table t1 using

[jira] [Created] (SPARK-36282) Spark ThriftServer does not search hive external table which storage in hbase

2021-07-25 Thread Yu Wang (Jira)
Yu Wang created SPARK-36282: --- Summary: Spark ThriftServer does not search hive external table which storage in hbase Key: SPARK-36282 URL: https://issues.apache.org/jira/browse/SPARK-36282 Project: Spark

[jira] [Commented] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386980#comment-17386980 ] Takeshi Yamamuro commented on SPARK-36218: -- Ah, okay. I got it now. Thanks. >

[jira] [Commented] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386979#comment-17386979 ] Takeshi Yamamuro commented on SPARK-36218: -- Ah, okay. Is it okay to close this

[jira] [Issue Comment Deleted] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-36218: - Comment: was deleted (was: Ah, okay. Is it okay to close this as resolved?) > Flaky Tes

[jira] [Commented] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386976#comment-17386976 ] Hyukjin Kwon commented on SPARK-36218: -- Oh actually I think this is now fixed via S

[jira] [Resolved] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36218. -- Resolution: Duplicate > Flaky Test: TPC-DS in PR builder > >

[jira] [Commented] (SPARK-36218) Flaky Test: TPC-DS in PR builder

2021-07-25 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386972#comment-17386972 ] Takeshi Yamamuro commented on SPARK-36218: -- Sorry for my late work.., has anyon