[jira] [Resolved] (SPARK-28720) Update AppVeyor R version to 3.6.1

2019-08-13 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28720. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25441

[jira] [Resolved] (SPARK-28719) Enable Github Actions for building master

2019-08-13 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28719. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25440

[jira] [Created] (SPARK-28719) Enable Github Actions for building master

2019-08-13 Thread DB Tsai (JIRA)
DB Tsai created SPARK-28719: --- Summary: Enable Github Actions for building master Key: SPARK-28719 URL: https://issues.apache.org/jira/browse/SPARK-28719 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-28606) Update CRAN key to recover docker image generation

2019-08-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28606. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25339

[jira] [Updated] (SPARK-26152) Synchronize Worker Cleanup with Worker Shutdown

2019-07-30 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26152: Affects Version/s: 2.4.0 2.4.1 2.4.2

[jira] [Updated] (SPARK-26152) Synchronize Worker Cleanup with Worker Shutdown

2019-07-30 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26152: Fix Version/s: 2.4.4 > Synchronize Worker Cleanup with Worker Shutdown >

[jira] [Updated] (SPARK-28157) Make SHS clear KVStore LogInfo for the blacklisted entries

2019-06-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-28157: Fix Version/s: 2.4.4 2.3.4 > Make SHS clear KVStore LogInfo for the blacklisted

[jira] [Resolved] (SPARK-28157) Make SHS clear KVStore LogInfo for the blacklisted entries

2019-06-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28157. - Resolution: Resolved Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 > Make SHS clear

[jira] [Resolved] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-06-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27100. - Resolution: Fixed Fix Version/s: 2.4.4 Issue resolved by pull request 24957

[jira] [Resolved] (SPARK-28118) Add `spark.eventLog.compression.codec` configuration

2019-06-20 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-28118. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24921

[jira] [Assigned] (SPARK-27838) Support user provided non-nullable avro schema for nullable catalyst schema without any null record

2019-05-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-27838: --- Assignee: DB Tsai > Support user provided non-nullable avro schema for nullable catalyst schema >

[jira] [Resolved] (SPARK-27838) Support user provided non-nullable avro schema for nullable catalyst schema without any null record

2019-05-24 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27838. - Resolution: Fixed Issue resolved by pull request 24682 [https://github.com/apache/spark/pull/24682] >

[jira] [Created] (SPARK-27838) Support user provided non-nullable avro schema for nullable catalyst schema without any null record

2019-05-24 Thread DB Tsai (JIRA)
DB Tsai created SPARK-27838: --- Summary: Support user provided non-nullable avro schema for nullable catalyst schema without any null record Key: SPARK-27838 URL: https://issues.apache.org/jira/browse/SPARK-27838

[jira] [Resolved] (SPARK-27762) Support user provided avro schema for writing fields with different ordering

2019-05-21 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27762. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24635

[jira] [Created] (SPARK-27762) Support user provided avro schema for writing fields with different ordering

2019-05-17 Thread DB Tsai (JIRA)
DB Tsai created SPARK-27762: --- Summary: Support user provided avro schema for writing fields with different ordering Key: SPARK-27762 URL: https://issues.apache.org/jira/browse/SPARK-27762 Project: Spark

[jira] [Updated] (SPARK-27646) Required refactoring for bytecode analysis

2019-05-06 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-27646: Summary: Required refactoring for bytecode analysis (was: Required refactoring for bytecode analysis

[jira] [Created] (SPARK-27646) Required refactoring for bytecode analysis work

2019-05-06 Thread DB Tsai (JIRA)
DB Tsai created SPARK-27646: --- Summary: Required refactoring for bytecode analysis work Key: SPARK-27646 URL: https://issues.apache.org/jira/browse/SPARK-27646 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-27455) spark-submit and friends should allow main artifact to be specified as a package

2019-04-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-27455: --- Assignee: Brian Lindblom > spark-submit and friends should allow main artifact to be specified as

[jira] [Updated] (SPARK-27094) Thread interrupt being swallowed while launching executors in YarnAllocator

2019-03-25 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-27094: Fix Version/s: 2.4.1 > Thread interrupt being swallowed while launching executors in YarnAllocator >

[jira] [Resolved] (SPARK-27197) Add ReadNestedSchemaTest for file-based data sources

2019-03-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27197. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24139

[jira] [Updated] (SPARK-25603) Generalize Nested Column Pruning

2019-03-15 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Summary: Generalize Nested Column Pruning (was: Nested Column Pruning) > Generalize Nested Column

[jira] [Updated] (SPARK-25603) Project Pushdown for Nested fields through various operators

2019-03-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Summary: Project Pushdown for Nested fields through various operators (was: `Projection` expression

[jira] [Updated] (SPARK-25603) Nested Column Pruning

2019-03-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Summary: Nested Column Pruning (was: Project Pushdown for Nested fields through various operators) >

[jira] [Commented] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

2019-03-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792390#comment-16792390 ] DB Tsai commented on SPARK-27026: - Issue resolved by pull request 23932

[jira] [Resolved] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

2019-03-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27026. - Resolution: Resolved > Upgrade Docker image for release build to Ubuntu 18.04 >

[jira] [Assigned] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

2019-03-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-27026: --- Assignee: DB Tsai > Upgrade Docker image for release build to Ubuntu 18.04 >

[jira] [Resolved] (SPARK-27123) Improve CollapseProject to handle projects cross limit/repartition/sample

2019-03-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27123. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24049

[jira] [Commented] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-10 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788862#comment-16788862 ] DB Tsai commented on SPARK-27097: - [~irashid] Initially, I was thinking this fix is for the platform

[jira] [Resolved] (SPARK-27097) Avoid embedding platform-dependent offsets literally in whole-stage generated code

2019-03-09 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-27097. - Resolution: Fixed Fix Version/s: 2.4.1 Issue resolved by pull request 24032

[jira] [Created] (SPARK-27026) Upgrade Docker image for release build to Ubuntu 18.04

2019-03-01 Thread DB Tsai (JIRA)
DB Tsai created SPARK-27026: --- Summary: Upgrade Docker image for release build to Ubuntu 18.04 Key: SPARK-27026 URL: https://issues.apache.org/jira/browse/SPARK-27026 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26958) Add NestedSchemaPruningBenchmark

2019-02-21 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26958. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23862

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2019-02-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761312#comment-16761312 ] DB Tsai commented on SPARK-17636: - [~MasterDDT] This is different from [SPARK-4502]. One is predicate

[jira] [Resolved] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26706. - Resolution: Resolved > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Affects Version/s: (was: 3.0.0) > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Affects Version/s: (was: 2.4.1) (was: 2.3.3) 2.3.2

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Fix Version/s: (was: 2.3.4) 2.3.3 > Fix Cast$mayTruncate for bytes >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Fix Version/s: 2.4.1 2.3.4 > Fix Cast$mayTruncate for bytes >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Priority: Blocker (was: Critical) > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Labels: correctness (was: ) > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Updated] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26706: Priority: Critical (was: Major) > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Assigned] (SPARK-26706) Fix Cast$mayTruncate for bytes

2019-01-23 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-26706: --- Assignee: Anton Okolnychyi > Fix Cast$mayTruncate for bytes > -- > >

[jira] [Assigned] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

2019-01-16 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-26619: --- Assignee: Liang-Chi Hsieh > Prune the unused serializers from `SerializeFromObject` >

[jira] [Resolved] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

2019-01-16 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26619. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23562

[jira] [Updated] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

2019-01-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26619: Affects Version/s: (was: 2.4.0) 3.0.0 > Prune the unused serializers from

[jira] [Updated] (SPARK-25603) `Projection` expression pushdown through `coalesce` and `limit`

2019-01-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25603: Affects Version/s: (was: 2.4.1) 3.0.0 > `Projection` expression pushdown

[jira] [Created] (SPARK-26619) Prune the unused serializers from `SerializeFromObject`

2019-01-14 Thread DB Tsai (JIRA)
DB Tsai created SPARK-26619: --- Summary: Prune the unused serializers from `SerializeFromObject` Key: SPARK-26619 URL: https://issues.apache.org/jira/browse/SPARK-26619 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26551) Selecting one complex field and having is null predicate on another complex field can cause error

2019-01-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26551. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-26551) Selecting one complex field and having is null predicate on another complex field can cause error

2019-01-07 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26551: Affects Version/s: 2.4.0 > Selecting one complex field and having is null predicate on another complex >

[jira] [Assigned] (SPARK-26551) Selecting one complex field and having is null predicate on another complex field can cause error

2019-01-07 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-26551: --- Assignee: Liang-Chi Hsieh > Selecting one complex field and having is null predicate on another

[jira] [Updated] (SPARK-26402) Accessing nested fields with different cases in case insensitive mode

2018-12-21 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26402: Description: {{GetStructField}} with different optional names should be semantically equal. We will use

[jira] [Updated] (SPARK-26402) Accessing nested fields with different cases in case insensitive mode

2018-12-21 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26402: Description: GetStructField with different optional names should be semantically equal. We will use this

[jira] [Resolved] (SPARK-26416) Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala`

2018-12-21 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26416. - Resolution: Won't Do > Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala` >

[jira] [Created] (SPARK-26416) Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala`

2018-12-19 Thread DB Tsai (JIRA)
DB Tsai created SPARK-26416: --- Summary: Refactor `ColumnPruning` from `Optimizer.scala` to `ColumnPruning.scala` Key: SPARK-26416 URL: https://issues.apache.org/jira/browse/SPARK-26416 Project: Spark

[jira] [Updated] (SPARK-26402) Canonicalization on GetStructField

2018-12-18 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26402: Summary: Canonicalization on GetStructField (was: GetStructField with different optional names are

[jira] [Updated] (SPARK-26402) Canonicalization on GetStructField

2018-12-18 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26402: Description: GetStructField with different optional names should be semantically equal. We will use this

[jira] [Created] (SPARK-26402) GetStructField with different optional names are semantically equal

2018-12-18 Thread DB Tsai (JIRA)
DB Tsai created SPARK-26402: --- Summary: GetStructField with different optional names are semantically equal Key: SPARK-26402 URL: https://issues.apache.org/jira/browse/SPARK-26402 Project: Spark

[jira] [Resolved] (SPARK-26133) Remove deprecated OneHotEncoder and rename OneHotEncoderEstimator to OneHotEncoder

2018-11-28 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-26133. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23100

[jira] [Updated] (SPARK-26179) `map_concat` should replace the value in the left side

2018-11-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26179: Description: Spark SQL map is internally represented by two arrays, and when two maps are concatenated,

[jira] [Updated] (SPARK-26179) `map_concat` should replace the value in the left side

2018-11-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26179: Description: Spark SQL map is internally represented by two arrays, and when two maps are concatenated,

[jira] [Updated] (SPARK-26179) `map_concat` should replace the value in the left side

2018-11-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26179: Description: See the following example,   {noformat} import org.apache.spark.sql.functions.udf val

[jira] [Created] (SPARK-26179) `map_concat` should replace the value in the left side

2018-11-26 Thread DB Tsai (JIRA)
DB Tsai created SPARK-26179: --- Summary: `map_concat` should replace the value in the left side Key: SPARK-26179 URL: https://issues.apache.org/jira/browse/SPARK-26179 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-26179) `map_concat` should replace the value in the left side

2018-11-26 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-26179: Description: See the following example,   {code:scala} import org.apache.spark.sql.functions.udf val

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK11

2018-11-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680117#comment-16680117 ] DB Tsai commented on SPARK-24421: - [~srowen] Great news! Thanks for looking at this. Out of my

[jira] [Updated] (SPARK-24421) sun.misc.Unsafe in JDK11

2018-11-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24421: Summary: sun.misc.Unsafe in JDK11 (was: sun.misc.Unsafe in JDK9+) > sun.misc.Unsafe in JDK11 >

[jira] [Updated] (SPARK-24422) Add JDK11 in our Jenkins' build servers

2018-11-06 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24422: Summary: Add JDK11 in our Jenkins' build servers (was: Add JDK9+ in our Jenkins' build servers) > Add

[jira] [Created] (SPARK-25956) Make Scala 2.12 as default Scala version in Spark 3.0

2018-11-06 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25956: --- Summary: Make Scala 2.12 as default Scala version in Spark 3.0 Key: SPARK-25956 URL: https://issues.apache.org/jira/browse/SPARK-25956 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK11

2018-11-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676210#comment-16676210 ] DB Tsai commented on SPARK-24417: - Scala community is working on official support for JDK11

[jira] [Resolved] (SPARK-25946) Upgrade ASM to 7.x to support JDK11

2018-11-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25946. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22953

[jira] [Assigned] (SPARK-25946) Upgrade ASM to 7.x to support JDK11

2018-11-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25946: --- Assignee: DB Tsai > Upgrade ASM to 7.x to support JDK11 > --- > >

[jira] [Created] (SPARK-25946) Upgrade ASM to 7.x to support JDK11

2018-11-05 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25946: --- Summary: Upgrade ASM to 7.x to support JDK11 Key: SPARK-25946 URL: https://issues.apache.org/jira/browse/SPARK-25946 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-25764) Avoid usage of deprecated methods in examples for BisectingKMeans

2018-11-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25764. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22786

[jira] [Resolved] (SPARK-25758) Deprecate BisectingKMeans compute cost

2018-11-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25758. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22869

[jira] [Updated] (SPARK-24417) Build and Run Spark on JDK11

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24417: Description: This is an umbrella JIRA for Apache Spark to support JDK11 As JDK8 is reaching EOL, and

[jira] [Updated] (SPARK-24417) Build and Run Spark on JDK11

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-24417: Summary: Build and Run Spark on JDK11 (was: Build and Run Spark on JDK9+) > Build and Run Spark on JDK11

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK9+

2018-11-02 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673769#comment-16673769 ] DB Tsai commented on SPARK-24417: - Since JDK 11 is LTS, should we skip 9 and 10? If no one is against

[jira] [Resolved] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

2018-10-31 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25860. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22857

[jira] [Commented] (SPARK-25889) Dynamic allocation load-aware ramp up

2018-10-30 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669384#comment-16669384 ] DB Tsai commented on SPARK-25889: - The problem sounds valid, and the solution could work. Since I'm not 

[jira] [Commented] (SPARK-25879) Schema pruning fails when a nested field and top level field are selected

2018-10-30 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669170#comment-16669170 ] DB Tsai commented on SPARK-25879: - I can confirm that with PR from SPARK-25407, this issue will be

[jira] [Created] (SPARK-25879) Schema pruning fails when a nested field and top level field are selected

2018-10-30 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25879: --- Summary: Schema pruning fails when a nested field and top level field are selected Key: SPARK-25879 URL: https://issues.apache.org/jira/browse/SPARK-25879 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25783: Comment: was deleted (was: cc seems it's related to SPARK-24715 which [~viirya] and [~dongjoon] 

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657619#comment-16657619 ] DB Tsai commented on SPARK-25783: - cc seems it's related to SPARK-24715 which [~viirya] and [~dongjoon] 

[jira] [Assigned] (SPARK-25662) Refactor DataSourceReadBenchmark to use main method

2018-10-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25662: --- Assignee: Peter Toth > Refactor DataSourceReadBenchmark to use main method >

[jira] [Resolved] (SPARK-25662) Refactor DataSourceReadBenchmark to use main method

2018-10-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25662. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22664

[jira] [Assigned] (SPARK-25699) Partially push down conjunctive predicated in Orc

2018-10-10 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25699: --- Assignee: Gengliang Wang > Partially push down conjunctive predicated in Orc >

[jira] [Resolved] (SPARK-25699) Partially push down conjunctive predicated in Orc

2018-10-10 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25699. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22684

[jira] [Created] (SPARK-25603) `Projection` expression pushdown through `coalesce` and `limit`

2018-10-03 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25603: --- Summary: `Projection` expression pushdown through `coalesce` and `limit` Key: SPARK-25603 URL: https://issues.apache.org/jira/browse/SPARK-25603 Project: Spark Issue

[jira] [Commented] (SPARK-25559) Just remove the unsupported predicates in Parquet

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631182#comment-16631182 ] DB Tsai commented on SPARK-25559: - https://github.com/apache/spark/pull/22574 > Just remove the

[jira] [Issue Comment Deleted] (SPARK-25556) Predicate Pushdown for Nested fields

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25556: Comment: was deleted (was: User 'dbtsai' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-25556) Predicate Pushdown for Nested fields

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25556: Comment: was deleted (was: User 'dbtsai' has created a pull request for this issue:

[jira] [Updated] (SPARK-25559) Just remove the unsupported predicates in Parquet

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25559: Description: Currently, in *ParquetFilters*, if one of the children predicates is not supported by

[jira] [Updated] (SPARK-25558) Pushdown predicates for nested fields in DataSource Strategy

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25558: Summary: Pushdown predicates for nested fields in DataSource Strategy (was: Creating predicates for

[jira] [Created] (SPARK-25559) Just remove the unsupported predicates in Parquet

2018-09-27 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25559: --- Summary: Just remove the unsupported predicates in Parquet Key: SPARK-25559 URL: https://issues.apache.org/jira/browse/SPARK-25559 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-25558) Creating predicates for nested fields in DataSource Strategy

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-25558: Issue Type: Sub-task (was: New Feature) Parent: SPARK-25556 > Creating predicates for nested

[jira] [Assigned] (SPARK-25558) Creating predicates for nested fields in DataSource Strategy

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25558: --- Assignee: DB Tsai > Creating predicates for nested fields in DataSource Strategy >

[jira] [Created] (SPARK-25558) Creating predicates for nested fields in DataSource Strategy

2018-09-27 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25558: --- Summary: Creating predicates for nested fields in DataSource Strategy Key: SPARK-25558 URL: https://issues.apache.org/jira/browse/SPARK-25558 Project: Spark Issue

[jira] [Created] (SPARK-25557) ORC predicate pushdown for nested fields

2018-09-27 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25557: --- Summary: ORC predicate pushdown for nested fields Key: SPARK-25557 URL: https://issues.apache.org/jira/browse/SPARK-25557 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-25556) Predicate Pushdown for Nested fields

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-25556: --- Assignee: DB Tsai > Predicate Pushdown for Nested fields > >

[jira] [Updated] (SPARK-17636) Parquet predicate pushdown for nested fields

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-17636: Summary: Parquet predicate pushdown for nested fields (was: Parquet filter push down doesn't handle

[jira] [Created] (SPARK-25556) Predicate Pushdown for Nested fields

2018-09-27 Thread DB Tsai (JIRA)
DB Tsai created SPARK-25556: --- Summary: Predicate Pushdown for Nested fields Key: SPARK-25556 URL: https://issues.apache.org/jira/browse/SPARK-25556 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2018-09-27 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-17636: Issue Type: Sub-task (was: Bug) Parent: SPARK-25556 > Parquet filter push down doesn't handle

<    1   2   3   4   5   6   7   >