[jira] [Commented] (SPARK-44212) Upgrade netty dependencies to 4.1.94.Final due to CVE-2023-34462

2023-06-27 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17737755#comment-17737755 ] Kazuaki Ishizaki commented on SPARK-44212: -- [https://github.com/apache/spark/pu

[jira] [Created] (SPARK-33590) Missing submenus for Performance Tuning in Spark SQL Guide

2020-11-29 Thread Kazuaki Ishizaki (Jira)
Kazuaki Ishizaki created SPARK-33590: Summary: Missing submenus for Performance Tuning in Spark SQL Guide Key: SPARK-33590 URL: https://issues.apache.org/jira/browse/SPARK-33590 Project: Spark

[jira] [Updated] (SPARK-33590) Missing submenus for Performance Tuning in Spark SQL Guide

2020-11-29 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-33590: - Attachment: image-2020-11-30-00-04-07-969.png > Missing submenus for Performance Tuning

[jira] [Updated] (SPARK-33590) Missing submenus for Performance Tuning in Spark SQL Guide

2020-11-29 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-33590: - Description: Sub-menus for \{Coalesce Hints for SQL Queries} and {Adaptive Query Executi

[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

2020-09-02 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17189784#comment-17189784 ] Kazuaki Ishizaki commented on SPARK-32312: -- I think that [this|https://github.c

[jira] [Commented] (SPARK-31538) Backport SPARK-25338 Ensure to call super.beforeAll() and super.afterAll() in test cases

2020-04-30 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096707#comment-17096707 ] Kazuaki Ishizaki commented on SPARK-31538: -- Do we want to port this to 2.4.6 an

[jira] [Commented] (SPARK-31538) Backport SPARK-25338 Ensure to call super.beforeAll() and super.afterAll() in test cases

2020-04-24 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091699#comment-17091699 ] Kazuaki Ishizaki commented on SPARK-31538: -- We could backport this. On the othe

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2020-03-18 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061914#comment-17061914 ] Kazuaki Ishizaki commented on SPARK-25728: -- For now, no update on my side. I a

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

2020-03-12 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058174#comment-17058174 ] Kazuaki Ishizaki commented on SPARK-25987: -- Sorry for coming here late since I

[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-12 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035464#comment-17035464 ] Kazuaki Ishizaki commented on SPARK-30711: -- The reason to show the exception as

[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-08 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033112#comment-17033112 ] Kazuaki Ishizaki commented on SPARK-30711: -- [~schreiber] Sorry, I made a mistak

[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-05 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031276#comment-17031276 ] Kazuaki Ishizaki commented on SPARK-30711: -- Now, I am looking at this with mast

[jira] [Comment Edited] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-04 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029724#comment-17029724 ] Kazuaki Ishizaki edited comment on SPARK-30711 at 2/4/20 10:20 AM: ---

[jira] [Comment Edited] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-04 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029724#comment-17029724 ] Kazuaki Ishizaki edited comment on SPARK-30711 at 2/4/20 10:19 AM: ---

[jira] [Commented] (SPARK-30711) 64KB JVM bytecode limit - janino.InternalCompilerException

2020-02-04 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029724#comment-17029724 ] Kazuaki Ishizaki commented on SPARK-30711: -- In my environment, both v3.0.0-prev

[jira] [Commented] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or 64K constant pool entry limit

2020-02-03 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028784#comment-17028784 ] Kazuaki Ishizaki commented on SPARK-22510: -- [~schreiber] Thank you for reportin

[jira] [Comment Edited] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-09-02 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920594#comment-16920594 ] Kazuaki Ishizaki edited comment on SPARK-28906 at 9/2/19 8:48 AM:

[jira] [Commented] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-09-01 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920594#comment-16920594 ] Kazuaki Ishizaki commented on SPARK-28906: -- For the information on {{git}} coma

[jira] [Commented] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-08-30 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919893#comment-16919893 ] Kazuaki Ishizaki commented on SPARK-28906: -- For user name, we have to pass {{US

[jira] [Comment Edited] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-08-30 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919420#comment-16919420 ] Kazuaki Ishizaki edited comment on SPARK-28906 at 8/30/19 10:51 AM: --

[jira] [Commented] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-08-30 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919420#comment-16919420 ] Kazuaki Ishizaki commented on SPARK-28906: -- In {{jars/spark-core_2.11-2.3.*.jar

[jira] [Commented] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-08-30 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919330#comment-16919330 ] Kazuaki Ishizaki commented on SPARK-28906: -- I attached output of 2.3.0 and 2.3.

[jira] [Updated] (SPARK-28891) do-release-docker.sh in master does not work for branch-2.3

2019-08-27 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-28891: - Description: According to [~maropu], [do-release-docker.sh|https://github.com/apache/sp

[jira] [Created] (SPARK-28891) do-release-docker.sh in master does not work for branch-2.3

2019-08-27 Thread Kazuaki Ishizaki (Jira)
Kazuaki Ishizaki created SPARK-28891: Summary: do-release-docker.sh in master does not work for branch-2.3 Key: SPARK-28891 URL: https://issues.apache.org/jira/browse/SPARK-28891 Project: Spark

[jira] [Commented] (SPARK-28699) Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-19 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910983#comment-16910983 ] Kazuaki Ishizaki commented on SPARK-28699: -- [~dongjoon] Thank you for pointing

[jira] [Commented] (SPARK-28699) Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-19 Thread Kazuaki Ishizaki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910907#comment-16910907 ] Kazuaki Ishizaki commented on SPARK-28699: -- [~smilegator] Thank you for cc. I w

[jira] [Commented] (SPARK-27761) Make UDF nondeterministic by default(?)

2019-05-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844396#comment-16844396 ] Kazuaki Ishizaki commented on SPARK-27761: -- IMHO, it looks good. It is fine for

[jira] [Created] (SPARK-27752) Updata lz4-java from 1.5.2 to 1.6.0

2019-05-16 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-27752: Summary: Updata lz4-java from 1.5.2 to 1.6.0 Key: SPARK-27752 URL: https://issues.apache.org/jira/browse/SPARK-27752 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-27752) Updata lz4-java from 1.5.1 to 1.6.0

2019-05-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-27752: - Summary: Updata lz4-java from 1.5.1 to 1.6.0 (was: Updata lz4-java from 1.5.2 to 1.6.0)

[jira] [Commented] (SPARK-27684) Reduce ScalaUDF conversion overheads for primitives

2019-05-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838229#comment-16838229 ] Kazuaki Ishizaki commented on SPARK-27684: -- Interesting idea > Reduce ScalaUDF

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819181#comment-16819181 ] Kazuaki Ishizaki commented on SPARK-27396: -- I have one question regarding low-l

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16817152#comment-16817152 ] Kazuaki Ishizaki commented on SPARK-27396: -- Thank you for sharing low-level API

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815812#comment-16815812 ] Kazuaki Ishizaki commented on SPARK-27396: -- Sorry for being late to comment sin

[jira] [Created] (SPARK-27397) Take care of OpenJ9 in JVM dependant parts

2019-04-06 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-27397: Summary: Take care of OpenJ9 in JVM dependant parts Key: SPARK-27397 URL: https://issues.apache.org/jira/browse/SPARK-27397 Project: Spark Issue Type

[jira] [Created] (SPARK-26508) Address warning messages in Java by lgtm.com

2018-12-31 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-26508: Summary: Address warning messages in Java by lgtm.com Key: SPARK-26508 URL: https://issues.apache.org/jira/browse/SPARK-26508 Project: Spark Issue Ty

[jira] [Commented] (SPARK-26463) Use ConfigEntry for hardcoded configs for scheduler categories.

2018-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730882#comment-16730882 ] Kazuaki Ishizaki commented on SPARK-26463: -- I will work for this > Use ConfigE

[jira] [Commented] (SPARK-26442) Use ConfigEntry for hardcoded configs.

2018-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730883#comment-16730883 ] Kazuaki Ishizaki commented on SPARK-26442: -- Thank you for updaing them. > Use

[jira] [Comment Edited] (SPARK-26442) Use ConfigEntry for hardcoded configs.

2018-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730883#comment-16730883 ] Kazuaki Ishizaki edited comment on SPARK-26442 at 12/30/18 4:18 AM: --

[jira] [Commented] (SPARK-26477) Use ConfigEntry for hardcoded configs for unsafe category.

2018-12-28 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730340#comment-16730340 ] Kazuaki Ishizaki commented on SPARK-26477: -- I will work for this > Use ConfigE

[jira] [Resolved] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24498. -- Resolution: Won't Do > Add JDK compiler for runtime codegen >

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703556#comment-16703556 ] Kazuaki Ishizaki commented on SPARK-24498: -- I see. Let us close for now. We may

[jira] [Created] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-26212: Summary: Upgrade maven from 3.5.4 to 3.6.0 Key: SPARK-26212 URL: https://issues.apache.org/jira/browse/SPARK-26212 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24255) Require Java 8 in SparkR description

2018-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690437#comment-16690437 ] Kazuaki Ishizaki commented on SPARK-24255: -- I do not know an existing library t

[jira] [Commented] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674442#comment-16674442 ] Kazuaki Ishizaki commented on SPARK-25776: -- Issue resolved by pull request 2275

[jira] [Updated] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25776: - Fix Version/s: 3.0.0 > The disk write buffer size must be greater than 12. > ---

[jira] [Resolved] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25776. -- Resolution: Fixed Assignee: liuxian > The disk write buffer size must be greater

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663394#comment-16663394 ] Kazuaki Ishizaki commented on SPARK-25829: -- I am curious about behavior in othe

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663368#comment-16663368 ] Kazuaki Ishizaki commented on SPARK-25829: -- cc [~ueshin] > Duplicated map keys

[jira] [Commented] (SPARK-25824) Remove duplicated map entries in `showString`

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663361#comment-16663361 ] Kazuaki Ishizaki commented on SPARK-25824: -- cc [~ueshin] During the implementat

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649428#comment-16649428 ] Kazuaki Ishizaki commented on SPARK-25728: -- cc: [~viirya], [~cloud_fan], [~mgai

[jira] [Issue Comment Deleted] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - Comment: was deleted (was: This JIRA entry is to start a discussion about adding structu

[jira] [Updated] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - Description: This JIRA entry is to start a discussion about adding structure intermediat

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649422#comment-16649422 ] Kazuaki Ishizaki commented on SPARK-25728: -- This JIRA entry is to start a discu

[jira] [Updated] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - External issue URL: (was: https://docs.google.com/document/d/1Jzf56bxpMpSwsGV_hSzl9wQG

[jira] [Updated] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - External issue URL: https://docs.google.com/document/d/1Jzf56bxpMpSwsGV_hSzl9wQG22hyI731

[jira] [Updated] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - External issue ID: https://docs.google.com/document/d/1Jzf56bxpMpSwsGV_hSzl9wQG22hyI731M

[jira] [Created] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25728: Summary: SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code Key: SPARK-25728 URL: https://issues.apache.org/jira/browse/SPARK-25728

[jira] [Resolved] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

2018-10-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25497. -- Resolution: Fixed Fix Version/s: 3.0.0 > limit operation within whole stage cod

[jira] [Assigned] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

2018-10-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reassigned SPARK-25497: Assignee: Wenchen Fan > limit operation within whole stage codegen should not con

[jira] [Comment Edited] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634344#comment-16634344 ] Kazuaki Ishizaki edited comment on SPARK-25538 at 10/1/18 5:21 PM: ---

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634344#comment-16634344 ] Kazuaki Ishizaki commented on SPARK-25538: -- This test case does not print {{63}

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-09-30 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16633568#comment-16633568 ] Kazuaki Ishizaki commented on SPARK-25538: -- Thank you. I will check it tonight

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-09-28 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631605#comment-16631605 ] Kazuaki Ishizaki commented on SPARK-25538: -- Thank for upload a schema. While I

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-09-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629281#comment-16629281 ] Kazuaki Ishizaki commented on SPARK-25538: -- Hi [~Steven Rand], would it be poss

[jira] [Resolved] (SPARK-25487) Refactor PrimitiveArrayBenchmark

2018-09-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25487. -- Resolution: Fixed Assignee: Chenxiao Mao Fix Version/s: 2.5.0 Issue re

[jira] [Commented] (SPARK-25432) Consider if using standard getOrCreate from PySpark into JVM SparkSession would simplify code

2018-09-19 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621416#comment-16621416 ] Kazuaki Ishizaki commented on SPARK-25432: -- nit: description seems to be in {{e

[jira] [Issue Comment Deleted] (SPARK-25437) Using OpenHashMap replace HashMap improve Encoder Performance

2018-09-19 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25437: - Comment: was deleted (was: Is such a feature for major release, not for maintenance rele

[jira] [Commented] (SPARK-25437) Using OpenHashMap replace HashMap improve Encoder Performance

2018-09-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617102#comment-16617102 ] Kazuaki Ishizaki commented on SPARK-25437: -- Is such a feature for major release

[jira] [Created] (SPARK-25444) Refactor GenArrayData.genCodeToCreateArrayData() method

2018-09-16 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25444: Summary: Refactor GenArrayData.genCodeToCreateArrayData() method Key: SPARK-25444 URL: https://issues.apache.org/jira/browse/SPARK-25444 Project: Spark

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2018-09-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611717#comment-16611717 ] Kazuaki Ishizaki commented on SPARK-20184: -- In {{branch-2.4}}, we still see the

[jira] [Commented] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2018-09-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611633#comment-16611633 ] Kazuaki Ishizaki commented on SPARK-16196: -- [~cloud_fan] This PR in the Jira en

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2018-09-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611502#comment-16611502 ] Kazuaki Ishizaki commented on SPARK-20184: -- Although I created another JIRA ht

[jira] [Commented] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

2018-09-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611494#comment-16611494 ] Kazuaki Ishizaki commented on SPARK-16196: -- I see. I will check this. > Optimi

[jira] [Created] (SPARK-25388) checkEvaluation may miss incorrect nullable of DataType in the result

2018-09-09 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25388: Summary: checkEvaluation may miss incorrect nullable of DataType in the result Key: SPARK-25388 URL: https://issues.apache.org/jira/browse/SPARK-25388 Project

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

2018-09-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604120#comment-16604120 ] Kazuaki Ishizaki commented on SPARK-25317: -- When I have been investigating this

[jira] [Created] (SPARK-25338) Several tests miss calling super.afterAll() in their afterAll() method

2018-09-04 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25338: Summary: Several tests miss calling super.afterAll() in their afterAll() method Key: SPARK-25338 URL: https://issues.apache.org/jira/browse/SPARK-25338 Projec

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

2018-09-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602567#comment-16602567 ] Kazuaki Ishizaki commented on SPARK-25317: -- I confirmed this performance differ

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

2018-09-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602506#comment-16602506 ] Kazuaki Ishizaki commented on SPARK-25317: -- Let me run this on 2.3 and master.

[jira] [Updated] (SPARK-25317) MemoryBlock performance regression

2018-09-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25317: - Description: eThere is a performance regression when calculating hash code for UTF8Strin

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Description: Invoking {{ArraysOverlap}} function with non-nullable array type throws the

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Summary: ArraysOverlap may throw a CompileException (was: ArraysOverlap throws an Excep

[jira] [Created] (SPARK-25310) ArraysOverlap throws an Exception

2018-09-02 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25310: Summary: ArraysOverlap throws an Exception Key: SPARK-25310 URL: https://issues.apache.org/jira/browse/SPARK-25310 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25178) Directly ship the StructType objects of the keySchema / valueSchema for xxxHashMapGenerator

2018-08-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25178: - Summary: Directly ship the StructType objects of the keySchema / valueSchema for xxxHash

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

2018-08-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587897#comment-16587897 ] Kazuaki Ishizaki commented on SPARK-25178: -- [~rednaxelafx] Thank you for openin

[jira] [Updated] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25036: - Description: When compiling with sbt, the following errors occur: There are -two- three

[jira] [Comment Edited] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575137#comment-16575137 ] Kazuaki Ishizaki edited comment on SPARK-25036 at 8/9/18 5:05 PM:

[jira] [Reopened] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-25036: -- Another type of compilation error is found > Scala 2.12 issues: Compilation error with sb

[jira] [Commented] (SPARK-25059) Exception while executing an action on DataFrame that read Json

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575129#comment-16575129 ] Kazuaki Ishizaki commented on SPARK-25059: -- Thank you for reporting the issue.

[jira] [Updated] (SPARK-25041) genjavadoc-plugin_0.10 is not found with sbt in scala-2.12

2018-08-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25041: - Summary: genjavadoc-plugin_0.10 is not found with sbt in scala-2.12 (was: genjavadoc-pl

[jira] [Created] (SPARK-25041) genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12

2018-08-07 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25041: Summary: genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12 Key: SPARK-25041 URL: https://issues.apache.org/jira/browse/SPARK-25041 Project: Spark

[jira] [Created] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-06 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25036: Summary: Scala 2.12 issues: Compilation error with sbt Key: SPARK-25036 URL: https://issues.apache.org/jira/browse/SPARK-25036 Project: Spark Issue T

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570440#comment-16570440 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen][~skonto] Thank you for yo

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569621#comment-16569621 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen] I see. The following part

[jira] [Created] (SPARK-24962) refactor CodeGenerator.createUnsafeArray

2018-07-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24962: Summary: refactor CodeGenerator.createUnsafeArray Key: SPARK-24962 URL: https://issues.apache.org/jira/browse/SPARK-24962 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560600#comment-16560600 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~ericfchang] Thank you very much f

[jira] [Created] (SPARK-24956) Upgrade maven from 3.3.9 to 3.5.4

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24956: Summary: Upgrade maven from 3.3.9 to 3.5.4 Key: SPARK-24956 URL: https://issues.apache.org/jira/browse/SPARK-24956 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559987#comment-16559987 ] Kazuaki Ishizaki commented on SPARK-24895: -- I see. Thank you very much. At firs

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559974#comment-16559974 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~yhuai] Thank you. BTW, how can I

[jira] [Commented] (SPARK-24925) input bytesRead metrics fluctuate from time to time

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559972#comment-16559972 ] Kazuaki Ishizaki commented on SPARK-24925: -- Do we need a test case or which tes

[jira] [Commented] (SPARK-24841) Memory leak in converting spark dataframe to pandas dataframe

2018-07-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552231#comment-16552231 ] Kazuaki Ishizaki commented on SPARK-24841: -- Thank you for reporting an issue wi

  1   2   3   4   5   6   7   8   >