[jira] [Commented] (SPARK-25824) Remove duplicated map entries in `showString`

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663361#comment-16663361 ] Kazuaki Ishizaki commented on SPARK-25824: -- cc [~ueshin] During the implementat

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663368#comment-16663368 ] Kazuaki Ishizaki commented on SPARK-25829: -- cc [~ueshin] > Duplicated map keys

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

2018-10-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663394#comment-16663394 ] Kazuaki Ishizaki commented on SPARK-25829: -- I am curious about behavior in othe

[jira] [Resolved] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25776. -- Resolution: Fixed Assignee: liuxian > The disk write buffer size must be greater

[jira] [Updated] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25776: - Fix Version/s: 3.0.0 > The disk write buffer size must be greater than 12. > ---

[jira] [Commented] (SPARK-25776) The disk write buffer size must be greater than 12.

2018-11-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674442#comment-16674442 ] Kazuaki Ishizaki commented on SPARK-25776: -- Issue resolved by pull request 2275

[jira] [Commented] (SPARK-24255) Require Java 8 in SparkR description

2018-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690437#comment-16690437 ] Kazuaki Ishizaki commented on SPARK-24255: -- I do not know an existing library t

[jira] [Created] (SPARK-26212) Upgrade maven from 3.5.4 to 3.6.0

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-26212: Summary: Upgrade maven from 3.5.4 to 3.6.0 Key: SPARK-26212 URL: https://issues.apache.org/jira/browse/SPARK-26212 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703556#comment-16703556 ] Kazuaki Ishizaki commented on SPARK-24498: -- I see. Let us close for now. We may

[jira] [Resolved] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24498. -- Resolution: Won't Do > Add JDK compiler for runtime codegen >

[jira] [Created] (SPARK-22508) 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create()

2017-11-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22508: Summary: 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create() Key: SPARK-22508 URL: https://issues.apache.org/jira/browse/SPARK-22508 Project

[jira] [Commented] (SPARK-22510) Exceptions caused by 64KB JVM bytecode limit

2017-11-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16250839#comment-16250839 ] Kazuaki Ishizaki commented on SPARK-22510: -- [~smilegator] Thanks, good idea. I w

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-11-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21337: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > SQL which has large ‘case

[jira] [Commented] (SPARK-22517) NullPointerException in ShuffleExternalSorter.spill()

2017-11-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254706#comment-16254706 ] Kazuaki Ishizaki commented on SPARK-22517: -- Could you please post a program that

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22498: - Description: {{concat}} can throw an exception due to the 64KB JVM bytecode limit when th

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22498: - Summary: 64KB JVM bytecode limit problem with concat (was: 64KB JVM bytecode limit probl

[jira] [Created] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22549: Summary: 64KB JVM bytecode limit problem with concat_ws Key: SPARK-22549 URL: https://issues.apache.org/jira/browse/SPARK-22549 Project: Spark Issue

[jira] [Updated] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22549: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit pr

[jira] [Created] (SPARK-22550) 64KB JVM bytecode limit problem with elt

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22550: Summary: 64KB JVM bytecode limit problem with elt Key: SPARK-22550 URL: https://issues.apache.org/jira/browse/SPARK-22550 Project: Spark Issue Type:

[jira] [Updated] (SPARK-22550) 64KB JVM bytecode limit problem with elt

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22550: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit pr

[jira] [Created] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-20 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22570: Summary: Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances Key: SPARK-22570 URL: https://issues.apache.org/jira/browse/SPARK-22570

[jira] [Commented] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260363#comment-16260363 ] Kazuaki Ishizaki commented on SPARK-22570: -- I am working on this with SPARK-2250

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or constant pool entry limits

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Summary: Exceptions caused by 64KB JVM bytecode or constant pool entry limits (was: Exc

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or 64K constant pool entry limit

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Summary: Exceptions caused by 64KB JVM bytecode or 64K constant pool entry limit (was:

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or constant pool entry limits

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Description: Codegen can throw an exception due to the 64KB JVM bytecode or 64K constant

[jira] [Updated] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > Cast may create a lot of U

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object in generated code

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Summary: Create a lot of global variables to reuse an object in generated code (was: Cre

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Summary: Create a lot of global variables to reuse an object (was: Cast may create a lot

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object in generated code

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Description: Generated code for several operations such as {{Cast}}, {{RegExpReplace}}, a

[jira] [Commented] (SPARK-22595) flaky test: CastSuite.SPARK-22500: cast for struct should not generate codes beyond 64KB

2017-11-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264663#comment-16264663 ] Kazuaki Ishizaki commented on SPARK-22595: -- I see. I will look at this. > flaky

[jira] [Created] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22603: Summary: 64KB JVM bytecode limit problem with FormatString Key: SPARK-22603 URL: https://issues.apache.org/jira/browse/SPARK-22603 Project: Spark Iss

[jira] [Updated] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22603: - Description: {{FormatString}} can throw an exception due to the 64KB JVM bytecode limit w

[jira] [Updated] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22603: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit pr

[jira] [Created] (SPARK-22608) Add new API to CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22608: Summary: Add new API to CodeGeneration.splitExpressions() Key: SPARK-22608 URL: https://issues.apache.org/jira/browse/SPARK-22608 Project: Spark Issu

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Summary: Avoid code duplication regarding CodeGeneration.splitExpressions() (was: Add ne

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Priority: Minor (was: Major) > Avoid code duplication regarding CodeGeneration.splitExpr

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Issue Type: Improvement (was: Bug) > Avoid code duplication regarding CodeGeneration.spl

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267901#comment-16267901 ] Kazuaki Ishizaki commented on SPARK-22373: -- [~mshen] Thank you for reporting thi

[jira] [Created] (SPARK-22668) CodegenContext.splitExpression() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22668: Summary: CodegenContext.splitExpression() creates incorrect results with global variable arguments Key: SPARK-22668 URL: https://issues.apache.org/jira/browse/SPARK-22668

[jira] [Commented] (SPARK-22668) CodegenContext.splitExpression() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274128#comment-16274128 ] Kazuaki Ishizaki commented on SPARK-22668: -- I am working for this. > CodegenCon

[jira] [Updated] (SPARK-22668) CodegenContext.splitExpressions() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22668: - Summary: CodegenContext.splitExpressions() creates incorrect results with global variable

[jira] [Updated] (SPARK-22668) CodegenContext.splitExpressions() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22668: - Description: {{CodegenContext.splitExpressions()}} creates incorrect results with argumen

[jira] [Created] (SPARK-22688) Upgrade Janino version 0.3.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22688: Summary: Upgrade Janino version 0.3.8 Key: SPARK-22688 URL: https://issues.apache.org/jira/browse/SPARK-22688 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-22688) Upgrade Janino version 3.0.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Summary: Upgrade Janino version 3.0.8 (was: Upgrade Janino version 0.3.8) > Upgrade Jani

[jira] [Updated] (SPARK-22688) Upgrade Janino version 3.0.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Description: [Janino 3.0.8|https://janino-compiler.github.io/janino/changelog.html] inclu

[jira] [Updated] (SPARK-22688) Upgrade Janino version to 3.0.8

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Summary: Upgrade Janino version to 3.0.8 (was: Upgrade Janino version 3.0.8) > Upgrade

[jira] [Created] (SPARK-22704) Reduce # of mutable variables in Least and greatest

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22704: Summary: Reduce # of mutable variables in Least and greatest Key: SPARK-22704 URL: https://issues.apache.org/jira/browse/SPARK-22704 Project: Spark I

[jira] [Created] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22705: Summary: Reduce # of mutable variables in Case, Coalesce, and In Key: SPARK-22705 URL: https://issues.apache.org/jira/browse/SPARK-22705 Project: Spark

[jira] [Created] (SPARK-22746) Avoid the generation of useless mutable states by SortMergeJoin

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22746: Summary: Avoid the generation of useless mutable states by SortMergeJoin Key: SPARK-22746 URL: https://issues.apache.org/jira/browse/SPARK-22746 Project: Spar

[jira] [Updated] (SPARK-22746) Avoid the generation of useless mutable states by SortMergeJoin

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22746: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22692 > Avoid the generation of us

[jira] [Created] (SPARK-22747) Shorten lifetime of global variables used in HashAggregateExec

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22747: Summary: Shorten lifetime of global variables used in HashAggregateExec Key: SPARK-22747 URL: https://issues.apache.org/jira/browse/SPARK-22747 Project: Spark

[jira] [Created] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22774: Summary: Add compilation check for generated code in TPCDSQuerySuite Key: SPARK-22774 URL: https://issues.apache.org/jira/browse/SPARK-22774 Project: Spark

[jira] [Comment Edited] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289471#comment-16289471 ] Kazuaki Ishizaki edited comment on SPARK-22774 at 12/13/17 4:24 PM: ---

[jira] [Commented] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289471#comment-16289471 ] Kazuaki Ishizaki commented on SPARK-22774: -- I will commit a PR for this soon. >

[jira] [Created] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22776: Summary: Increase default value of spark.sql.codegen.maxFields Key: SPARK-22776 URL: https://issues.apache.org/jira/browse/SPARK-22776 Project: Spark

[jira] [Commented] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290351#comment-16290351 ] Kazuaki Ishizaki commented on SPARK-22776: -- Wait for re-merging SPARK-22600 > I

[jira] [Created] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22813: Summary: run-tests.py fails when /usr/sbin/lsof does not exist Key: SPARK-22813 URL: https://issues.apache.org/jira/browse/SPARK-22813 Project: Spark

[jira] [Updated] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22813: - Description: Running ./dev/run-tests.py for mvn on OS that does not have /usr/sbin/lsof

[jira] [Commented] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293570#comment-16293570 ] Kazuaki Ishizaki commented on SPARK-22813: -- Thank you for pointing the PR that w

[jira] [Created] (SPARK-22848) Do not execute generated code of functions in Generator

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22848: Summary: Do not execute generated code of functions in Generator Key: SPARK-22848 URL: https://issues.apache.org/jira/browse/SPARK-22848 Project: Spark

[jira] [Updated] (SPARK-22848) Do not execute generated code of functions in Generator

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-22692 > Do not execute gen

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Summary: Avoid the generation of useless mutable states by Stack function (was: Do not e

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for {{Stack}}.

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for {Stack}. (

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for {{{Stack}}}

[jira] [Created] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22868: Summary: 64KB JVM bytecode limit problem with aggregation Key: SPARK-22868 URL: https://issues.apache.org/jira/browse/SPARK-22868 Project: Spark Issu

[jira] [Created] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22869: Summary: 64KB JVM bytecode limit problem with filter Key: SPARK-22869 URL: https://issues.apache.org/jira/browse/SPARK-22869 Project: Spark Issue Typ

[jira] [Commented] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300945#comment-16300945 ] Kazuaki Ishizaki commented on SPARK-22868: -- I am working for this and waiting fo

[jira] [Commented] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300946#comment-16300946 ] Kazuaki Ishizaki commented on SPARK-22869: -- I am working for this and waiting fo

[jira] [Updated] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22868: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit pr

[jira] [Updated] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22869: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit pr

[jira] [Commented] (SPARK-22500) 64KB JVM bytecode limit problem with cast

2017-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306244#comment-16306244 ] Kazuaki Ishizaki commented on SPARK-22500: -- Thank you for your report. I think t

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code prints {{c=0}}. The value of {{c}} must be 2. {{ds}} must

[jira] [Created] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22935: Summary: Dataset with Java Beans for java.sql.Date produces incorrect result Key: SPARK-22935 URL: https://issues.apache.org/jira/browse/SPARK-22935 Project:

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307520#comment-16307520 ] Kazuaki Ishizaki commented on SPARK-22935: -- I am working for this. > Dataset wi

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307531#comment-16307531 ] Kazuaki Ishizaki commented on SPARK-21413: -- SPARK-22520 solved this issue. > Mu

[jira] [Resolved] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-21413. -- Resolution: Duplicate > Multiple projections with CASE WHEN fails to run generated code

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code can throw an exception with or without whole-stage codege

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308194#comment-16308194 ] Kazuaki Ishizaki commented on SPARK-22935: -- [~jlaskowski] When you see the sche

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333298#comment-16333298 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thank you for reporting an issue. Co

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333897#comment-16333897 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thanks, I will look at this. > Code

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336957#comment-16336957 ] Kazuaki Ishizaki commented on SPARK-18016: -- I cannot reproduce this issue using

[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336957#comment-16336957 ] Kazuaki Ishizaki edited comment on SPARK-18016 at 1/24/18 5:49 AM:

[jira] [Commented] (SPARK-24841) Memory leak in converting spark dataframe to pandas dataframe

2018-07-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552231#comment-16552231 ] Kazuaki Ishizaki commented on SPARK-24841: -- Thank you for reporting an issue wi

[jira] [Commented] (SPARK-24925) input bytesRead metrics fluctuate from time to time

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559972#comment-16559972 ] Kazuaki Ishizaki commented on SPARK-24925: -- Do we need a test case or which tes

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559974#comment-16559974 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~yhuai] Thank you. BTW, how can I

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559987#comment-16559987 ] Kazuaki Ishizaki commented on SPARK-24895: -- I see. Thank you very much. At firs

[jira] [Created] (SPARK-24956) Upgrade maven from 3.3.9 to 3.5.4

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24956: Summary: Upgrade maven from 3.3.9 to 3.5.4 Key: SPARK-24956 URL: https://issues.apache.org/jira/browse/SPARK-24956 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

2018-07-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560600#comment-16560600 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~ericfchang] Thank you very much f

[jira] [Created] (SPARK-24962) refactor CodeGenerator.createUnsafeArray

2018-07-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24962: Summary: refactor CodeGenerator.createUnsafeArray Key: SPARK-24962 URL: https://issues.apache.org/jira/browse/SPARK-24962 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569621#comment-16569621 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen] I see. The following part

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

2018-08-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570440#comment-16570440 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen][~skonto] Thank you for yo

[jira] [Created] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-06 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25036: Summary: Scala 2.12 issues: Compilation error with sbt Key: SPARK-25036 URL: https://issues.apache.org/jira/browse/SPARK-25036 Project: Spark Issue T

[jira] [Created] (SPARK-25041) genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12

2018-08-07 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-25041: Summary: genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12 Key: SPARK-25041 URL: https://issues.apache.org/jira/browse/SPARK-25041 Project: Spark

[jira] [Updated] (SPARK-25041) genjavadoc-plugin_0.10 is not found with sbt in scala-2.12

2018-08-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25041: - Summary: genjavadoc-plugin_0.10 is not found with sbt in scala-2.12 (was: genjavadoc-pl

[jira] [Commented] (SPARK-25059) Exception while executing an action on DataFrame that read Json

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575129#comment-16575129 ] Kazuaki Ishizaki commented on SPARK-25059: -- Thank you for reporting the issue.

[jira] [Reopened] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-25036: -- Another type of compilation error is found > Scala 2.12 issues: Compilation error with sb

[jira] [Comment Edited] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575137#comment-16575137 ] Kazuaki Ishizaki edited comment on SPARK-25036 at 8/9/18 5:05 PM:

[jira] [Updated] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

2018-08-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25036: - Description: When compiling with sbt, the following errors occur: There are -two- three

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

2018-08-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587897#comment-16587897 ] Kazuaki Ishizaki commented on SPARK-25178: -- [~rednaxelafx] Thank you for openin

<    2   3   4   5   6   7   8   9   10   >