[jira] [Resolved] (SPARK-27349) Dealing with TimeVars removed in Hive 2.x

2019-04-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27349. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Dealing with TimeVars

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.2

2019-04-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23710: Summary: Upgrade the built-in Hive to 2.3.4 for hadoop-3.2 (was: Upgrade the built-in Hive to 2.3.4 for

[jira] [Commented] (SPARK-27198) Heartbeat interval mismatch in driver and executor

2019-03-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806062#comment-16806062 ] Xiao Li commented on SPARK-27198: - [~srowen][~dbtsai] This requires a release note since it changes the

[jira] [Updated] (SPARK-27198) Heartbeat interval mismatch in driver and executor

2019-03-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27198: Fix Version/s: (was: 2.4.2) 2.4.1 > Heartbeat interval mismatch in driver and

[jira] [Updated] (SPARK-27198) Heartbeat interval mismatch in driver and executor

2019-03-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27198: Labels: release-notes (was: ) > Heartbeat interval mismatch in driver and executor >

[jira] [Resolved] (SPARK-27279) Reuse subquery should compare child node only

2019-03-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27279. - Resolution: Fixed Assignee: Adrian Wang Fix Version/s: 3.0.0 > Reuse subquery should

[jira] [Resolved] (SPARK-27182) Move the conflict source code of the sql/core module to sql/core/v1.2.1

2019-03-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27182. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Move the conflict source

[jira] [Resolved] (SPARK-24935) Problem with Executing Hive UDF's from Spark 2.2 Onwards

2019-03-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24935. - Resolution: Fixed Assignee: Parth Gandhi Fix Version/s: 3.0.0 2.4.1

[jira] [Resolved] (SPARK-27236) Refactor log-appender pattern in tests

2019-03-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27236. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 3.0.0 > Refactor log-appender

[jira] [Assigned] (SPARK-26990) Difference in handling of mixed-case partition column names after SPARK-26188

2019-03-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26990: --- Assignee: Gengliang Wang > Difference in handling of mixed-case partition column names after

[jira] [Commented] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791321#comment-16791321 ] Xiao Li commented on SPARK-25449: - This changed the unit of conf. > Don't send zero accumulators in

[jira] [Updated] (SPARK-25449) Don't send zero accumulators in heartbeats

2019-03-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25449: Labels: release-notes (was: ) > Don't send zero accumulators in heartbeats >

[jira] [Resolved] (SPARK-27119) Do not infer schema when reading Hive serde table with native data source

2019-03-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27119. - Resolution: Fixed Fix Version/s: 3.0.0 > Do not infer schema when reading Hive serde table with

[jira] [Commented] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787356#comment-16787356 ] Xiao Li commented on SPARK-27097: - The fix will be pushed soon. > Avoid embedding platform-dependent

[jira] [Commented] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787357#comment-16787357 ] Xiao Li commented on SPARK-27097: - This is not a regression but a long-standing issue. > Avoid

[jira] [Updated] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27097: Priority: Blocker (was: Major) > Avoid embedding platform-dependent offsets literally in whole-stage

[jira] [Updated] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27097: Priority: Critical (was: Blocker) > Avoid embedding platform-dependent offsets literally in whole-stage

[jira] [Updated] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27097: Description: Avoid embedding platform-dependent offsets literally in whole-stage generated code. Spark

[jira] [Updated] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27097: Description: Avoid embedding platform-dependent offsets literally in whole-stage generated code. Spark

[jira] [Created] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-07 Thread Xiao Li (JIRA)
Xiao Li created SPARK-27097: --- Summary: Avoid embedding platform-dependent offsets literally in whole-stage generated code Key: SPARK-27097 URL: https://issues.apache.org/jira/browse/SPARK-27097 Project:

[jira] [Commented] (SPARK-27093) Honor ParseMode in AvroFileFormat

2019-03-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787334#comment-16787334 ] Xiao Li commented on SPARK-27093: - [~hyukjin.kwon] Are you interested in this task? > Honor ParseMode

[jira] [Resolved] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-03-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26727. - Resolution: Not A Bug > CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException >

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-03-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784956#comment-16784956 ] Xiao Li commented on SPARK-26727: - I resolved the ticket as "Not a bug". This is kind of a well-known

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-03-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784949#comment-16784949 ] Xiao Li commented on SPARK-26727: - Hi, all, this could happen since the whole DDL are not atomic. For

[jira] [Resolved] (SPARK-26956) remove streaming output mode from data source v2 APIs

2019-03-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26956. - Resolution: Fixed Fix Version/s: 3.0.0 > remove streaming output mode from data source v2 APIs >

[jira] [Resolved] (SPARK-26785) data source v2 API refactor: streaming write

2019-02-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26785. - Resolution: Fixed Fix Version/s: 3.0.0 > data source v2 API refactor: streaming write >

[jira] [Resolved] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2019-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24570. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > SparkSQL - show

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2019-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24570: Issue Type: Improvement (was: New Feature) > SparkSQL - show schemas/tables in dropdowns of SQL client

[jira] [Created] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2019-02-16 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26905: --- Summary: Revisit reserved/non-reserved keywords based on the ANSI SQL standard Key: SPARK-26905 URL: https://issues.apache.org/jira/browse/SPARK-26905 Project: Spark

[jira] [Created] (SPARK-26904) API Auditing

2019-02-16 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26904: --- Summary: API Auditing Key: SPARK-26904 URL: https://issues.apache.org/jira/browse/SPARK-26904 Project: Spark Issue Type: Umbrella Components: PySpark, Spark

[jira] [Updated] (SPARK-26904) API Auditing

2019-02-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26904: Priority: Blocker (was: Major) > API Auditing > > > Key: SPARK-26904 >

[jira] [Resolved] (SPARK-26840) Avoid cost-based join reorder in presence of join hints

2019-02-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26840. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 3.0.0 > Avoid cost-based join

[jira] [Resolved] (SPARK-26861) deprecate typed sum/count/average

2019-02-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26861. - Resolution: Fixed Fix Version/s: 3.0.0 > deprecate typed sum/count/average >

[jira] [Commented] (SPARK-26862) assertion failed in ParquetRowConverter

2019-02-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768764#comment-16768764 ] Xiao Li commented on SPARK-26862: - [~CodingCat] It sounds like the problem is not an issue, right? >

[jira] [Created] (SPARK-26820) Issue Error/Warning when Hint is not applicable

2019-02-03 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26820: --- Summary: Issue Error/Warning when Hint is not applicable Key: SPARK-26820 URL: https://issues.apache.org/jira/browse/SPARK-26820 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26799) Make ANTLR v4 version consistent between Maven and SBT

2019-01-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26799. - Resolution: Fixed Assignee: Chenxiao Mao Fix Version/s: 3.0.0 > Make ANTLR v4 version

[jira] [Resolved] (SPARK-26695) data source V2 API refactoring (continuous read)

2019-01-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26695. - Resolution: Fixed Fix Version/s: 3.0.0 > data source V2 API refactoring (continuous read) >

[jira] [Commented] (SPARK-21287) Cannot use Int.MIN_VALUE as Spark SQL fetchsize

2019-01-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754318#comment-16754318 ] Xiao Li commented on SPARK-21287: - [~bestcastor] Feel free to submit a PR > Cannot use Int.MIN_VALUE

[jira] [Commented] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754300#comment-16754300 ] Xiao Li commented on SPARK-26708: - I do not think this affects 2.3 > Incorrect result caused by

[jira] [Commented] (SPARK-26741) Analyzer incorrectly resolves aggregate function outside of Aggregate operators

2019-01-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753748#comment-16753748 ] Xiao Li commented on SPARK-26741: - Could we re-enable the test by making the test case valid? That

[jira] [Resolved] (SPARK-26735) Verify plan integrity for special expressions

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26735. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 3.0.0 > Verify plan integrity for

[jira] [Commented] (SPARK-26630) Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753146#comment-16753146 ] Xiao Li commented on SPARK-26630: - This is an improvement. Thanks! > Support reading Hive-serde tables

[jira] [Resolved] (SPARK-26630) Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26630. - Resolution: Fixed Fix Version/s: 3.0.0 > Support reading Hive-serde tables whose INPUTFORMAT is

[jira] [Assigned] (SPARK-26630) Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26630: --- Assignee: Deegue > Support reading Hive-serde tables whose INPUTFORMAT is >

[jira] [Updated] (SPARK-26630) Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26630: Summary: Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce (was:

[jira] [Updated] (SPARK-26630) Support reading Hive-serde tables whose INPUTFORMAT is org.apache.hadoop.mapreduce

2019-01-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26630: Issue Type: Improvement (was: Bug) > Support reading Hive-serde tables whose INPUTFORMAT is >

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752652#comment-16752652 ] Xiao Li commented on SPARK-26727: - Are you using S3-like object store? These file systems do not

[jira] [Created] (SPARK-26725) Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26725: --- Summary: Fix the input values of UnifiedMemoryManager constructor in test suites Key: SPARK-26725 URL: https://issues.apache.org/jira/browse/SPARK-26725 Project: Spark

[jira] [Updated] (SPARK-26725) Fix the input values of UnifiedMemoryManager constructor in test suites

2019-01-24 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26725: Labels: starter (was: ) > Fix the input values of UnifiedMemoryManager constructor in test suites >

[jira] [Assigned] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26709: --- Assignee: Gengliang Wang (was: Xiao Li) > OptimizeMetadataOnlyQuery does not correctly handle the

[jira] [Assigned] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26709: --- Assignee: Xiao Li > OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

[jira] [Updated] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26709: Summary: OptimizeMetadataOnlyQuery does not correctly handle the files with zero record (was:

[jira] [Updated] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the empty files

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26709: Description: {code:java} import org.apache.spark.sql.functions.lit

[jira] [Updated] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the empty files

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26709: Description: {code:java} import org.apache.spark.sql.functions.lit

[jira] [Created] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the empty files

2019-01-23 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26709: --- Summary: OptimizeMetadataOnlyQuery does not correctly handle the empty files Key: SPARK-26709 URL: https://issues.apache.org/jira/browse/SPARK-26709 Project: Spark

[jira] [Updated] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26708: Description: When performing non-cascading cache invalidation, {{recache}}  is called on the other cache

[jira] [Updated] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26708: Labels: correctness (was: ) > Incorrect result caused by inconsistency between a SQL cache's cached RDD

[jira] [Created] (SPARK-26708) Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan

2019-01-23 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26708: --- Summary: Incorrect result caused by inconsistency between a SQL cache's cached RDD and its physical plan Key: SPARK-26708 URL: https://issues.apache.org/jira/browse/SPARK-26708

[jira] [Resolved] (SPARK-26520) data source V2 API refactoring (micro-batch read)

2019-01-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26520. - Resolution: Fixed Fix Version/s: 3.0.0 > data source V2 API refactoring (micro-batch read) >

[jira] [Updated] (SPARK-26664) Make DecimalType's minimum adjusted scale configurable

2019-01-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26664: Target Version/s: 3.0.0 > Make DecimalType's minimum adjusted scale configurable >

[jira] [Resolved] (SPARK-26659) Duplicate cmd.nodeName in the explain output of DataWritingCommandExec

2019-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26659. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 3.0.0 > Duplicate cmd.nodeName in

[jira] [Assigned] (SPARK-26659) Duplicate cmd.nodeName in the explain output of DataWritingCommandExec

2019-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26659: --- Assignee: Kris Mok (was: Xiao Li) > Duplicate cmd.nodeName in the explain output of

[jira] [Created] (SPARK-26649) Noop Streaming Sink using DSV2

2019-01-17 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26649: --- Summary: Noop Streaming Sink using DSV2 Key: SPARK-26649 URL: https://issues.apache.org/jira/browse/SPARK-26649 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-26648) Noop Streaming Sink using DSV2

2019-01-17 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26648: --- Summary: Noop Streaming Sink using DSV2 Key: SPARK-26648 URL: https://issues.apache.org/jira/browse/SPARK-26648 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-26622) Improve wording in SQLMetrics labels

2019-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26622. - Resolution: Fixed Assignee: Juliusz Sompolski Fix Version/s: 3.0.0 > Improve wording in

[jira] [Resolved] (SPARK-26633) Add ExecutorClassLoader.getResourceAsStream

2019-01-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26633. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 3.0.0 2.4.1 >

[jira] [Assigned] (SPARK-25530) data source v2 API refactor (batch write)

2019-01-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25530: --- Assignee: Wenchen Fan > data source v2 API refactor (batch write) >

[jira] [Resolved] (SPARK-25530) data source v2 API refactor (batch write)

2019-01-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25530. - Resolution: Fixed Fix Version/s: 3.0.0 > data source v2 API refactor (batch write) >

[jira] [Resolved] (SPARK-26576) Broadcast hint not applied to partitioned table

2019-01-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26576. - Resolution: Fixed Assignee: John Zhuge Fix Version/s: 3.0.0 2.4.1

[jira] [Resolved] (SPARK-26448) retain the difference between 0.0 and -0.0

2019-01-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26448. - Resolution: Fixed Fix Version/s: 3.0.0 > retain the difference between 0.0 and -0.0 >

[jira] [Created] (SPARK-26582) Update the optimizer rule NormalizeFloatingNumbers

2019-01-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26582: --- Summary: Update the optimizer rule NormalizeFloatingNumbers Key: SPARK-26582 URL: https://issues.apache.org/jira/browse/SPARK-26582 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-24196) Spark Thrift Server - SQL Client connections does't show db artefacts

2019-01-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24196. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 > Spark Thrift Server -

[jira] [Resolved] (SPARK-26065) Change query hint from a `LogicalPlan` to a field

2019-01-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26065. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 3.0.0 > Change query hint from a

[jira] [Resolved] (SPARK-26548) Don't block during query optimization

2019-01-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26548. - Resolution: Fixed Fix Version/s: 3.0.0 > Don't block during query optimization >

[jira] [Assigned] (SPARK-26548) Don't block during query optimization

2019-01-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26548: --- Assignee: Dave DeCaprio > Don't block during query optimization >

[jira] [Resolved] (SPARK-26545) Fix typo in EqualNullSafe's truth table comment

2019-01-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26545. - Resolution: Fixed Assignee: Kris Mok Fix Version/s: 3.0.0 2.4.1

[jira] [Updated] (SPARK-26215) define reserved keywords after SQL standard

2018-12-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26215: Target Version/s: 3.0.0 > define reserved keywords after SQL standard >

[jira] [Resolved] (SPARK-26409) SQLConf should be serializable in test sessions

2018-12-20 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26409. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > SQLConf should be

[jira] [Updated] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames for the entire partition

2018-12-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26412: Target Version/s: 3.0.0 > Allow Pandas UDF to take an iterator of pd.DataFrames for the entire partition

[jira] [Resolved] (SPARK-26366) Except with transform regression

2018-12-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26366. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Except with transform

[jira] [Assigned] (SPARK-26366) Except with transform regression

2018-12-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-26366: --- Assignee: Marco Gaido > Except with transform regression > > >

[jira] [Resolved] (SPARK-24933) SinkProgress should report written rows

2018-12-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24933. - Resolution: Fixed Assignee: Vaclav Kosar Fix Version/s: 3.0.0 > SinkProgress should

[jira] [Issue Comment Deleted] (SPARK-26222) Scan: track file listing time

2018-12-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26222: Comment: was deleted (was: xuanyuanking closed pull request #23298: [SPARK-26222][SQL] Track file

[jira] [Issue Comment Deleted] (SPARK-26222) Scan: track file listing time

2018-12-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26222: Comment: was deleted (was: xuanyuanking opened a new pull request #23327: [SPARK-26222][SQL] Track file

[jira] [Resolved] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

2018-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26368. - Resolution: Fixed Fix Version/s: 3.0.0 > Make it clear that getOrInferFileFormatSchema doesn't

[jira] [Created] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

2018-12-13 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26367: --- Summary: Remove ReplaceExceptWithFilter from nonExcludableRules Key: SPARK-26367 URL: https://issues.apache.org/jira/browse/SPARK-26367 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26098) Show associated SQL query in Job page

2018-12-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26098. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > Show associated SQL

[jira] [Created] (SPARK-26307) Fix CTAS when INSERT a partitioned table using Hive serde

2018-12-07 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26307: --- Summary: Fix CTAS when INSERT a partitioned table using Hive serde Key: SPARK-26307 URL: https://issues.apache.org/jira/browse/SPARK-26307 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-12-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Labels: release-notes (was: ) > Spark 2.4.0 Partitioning behavior breaks backwards compatibility >

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-12-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Labels: correctness release-notes (was: release-notes) > Spark 2.4.0 Partitioning behavior breaks

[jira] [Resolved] (SPARK-25573) Combine resolveExpression and resolve in the rule ResolveReferences

2018-12-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25573. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 3.0.0 > Combine

[jira] [Resolved] (SPARK-26241) Add queryId to IncrementalExecution

2018-12-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26241. - Resolution: Fixed Fix Version/s: 3.0.0 > Add queryId to IncrementalExecution >

[jira] [Resolved] (SPARK-25528) data source V2 API refactoring (batch read)

2018-11-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25528. - Resolution: Fixed Fix Version/s: 3.0.0 > data source V2 API refactoring (batch read) >

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Target Version/s: 2.4.1 > Spark 2.4.0 Partitioning behavior breaks backwards compatibility >

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Component/s: (was: Spark Core) SQL > Spark 2.4.0 Partitioning behavior breaks

[jira] [Updated] (SPARK-26188) Spark 2.4.0 Partitioning behavior breaks backwards compatibility

2018-11-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26188: Priority: Critical (was: Minor) > Spark 2.4.0 Partitioning behavior breaks backwards compatibility >

[jira] [Created] (SPARK-26193) Implement shuffle write metrics in SQL

2018-11-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26193: --- Summary: Implement shuffle write metrics in SQL Key: SPARK-26193 URL: https://issues.apache.org/jira/browse/SPARK-26193 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16700880#comment-16700880 ] Xiao Li commented on SPARK-26189: - cc [~huaxingao] > Fix the doc of unionAll in SparkR >

[jira] [Updated] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26189: Description: We should fix the doc of unionAll in SparkR. See the discussion:

[jira] [Created] (SPARK-26189) Fix the doc of unionAll in SparkR

2018-11-27 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26189: --- Summary: Fix the doc of unionAll in SparkR Key: SPARK-26189 URL: https://issues.apache.org/jira/browse/SPARK-26189 Project: Spark Issue Type: Documentation

<    4   5   6   7   8   9   10   11   12   13   >