[jira] [Updated] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25728: - External issue ID: https://docs.google.com/document/d

[jira] [Created] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

Kazuaki Ishizaki created SPARK-25728: Summary: SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code Key: SPARK-25728 URL: https://issues.apache.org/jira/browse/SPARK-25728

[jira] [Created] (ARROW-3476) [Java] mvn test in memory fails on a big-endian platform

Kazuaki Ishizaki created ARROW-3476: --- Summary: [Java] mvn test in memory fails on a big-endian platform Key: ARROW-3476 URL: https://issues.apache.org/jira/browse/ARROW-3476 Project: Apache Arrow

[jira] [Created] (ARROW-3476) [Java] mvn test in memory fails on a big-endian platform

Kazuaki Ishizaki created ARROW-3476: --- Summary: [Java] mvn test in memory fails on a big-endian platform Key: ARROW-3476 URL: https://issues.apache.org/jira/browse/ARROW-3476 Project: Apache Arrow

[jira] [Resolved] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25497. -- Resolution: Fixed Fix Version/s: 3.0.0 > limit operation within whole st

[jira] [Assigned] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reassigned SPARK-25497: Assignee: Wenchen Fan > limit operation within whole stage codegen sho

[jira] [Comment Edited] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634344#comment-16634344 ] Kazuaki Ishizaki edited comment on SPARK-25538 at 10/1/18 5:21 PM

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634344#comment-16634344 ] Kazuaki Ishizaki commented on SPARK-25538: -- This test case does not print {{63}}. {code

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633568#comment-16633568 ] Kazuaki Ishizaki commented on SPARK-25538: -- Thank you. I will check it tonight in Japan

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631605#comment-16631605 ] Kazuaki Ishizaki commented on SPARK-25538: -- Thank for upload a schema. While I looked

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629281#comment-16629281 ] Kazuaki Ishizaki commented on SPARK-25538: -- Hi [~Steven Rand], would it be possible to share

[jira] [Resolved] (SPARK-25487) Refactor PrimitiveArrayBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25487. -- Resolution: Fixed Assignee: Chenxiao Mao Fix Version/s: 2.5.0 Issue

[jira] [Commented] (SPARK-25432) Consider if using standard getOrCreate from PySpark into JVM SparkSession would simplify code

[ https://issues.apache.org/jira/browse/SPARK-25432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621416#comment-16621416 ] Kazuaki Ishizaki commented on SPARK-25432: -- nit: description seems to be in {{environment}} now

[jira] [Issue Comment Deleted] (SPARK-25437) Using OpenHashMap replace HashMap improve Encoder Performance

[ https://issues.apache.org/jira/browse/SPARK-25437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25437: - Comment: was deleted (was: Is such a feature for major release, not for maintenance

[jira] [Commented] (SPARK-25437) Using OpenHashMap replace HashMap improve Encoder Performance

[ https://issues.apache.org/jira/browse/SPARK-25437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617102#comment-16617102 ] Kazuaki Ishizaki commented on SPARK-25437: -- Is such a feature for major release

[jira] [Created] (SPARK-25444) Refactor GenArrayData.genCodeToCreateArrayData() method

Kazuaki Ishizaki created SPARK-25444: Summary: Refactor GenArrayData.genCodeToCreateArrayData() method Key: SPARK-25444 URL: https://issues.apache.org/jira/browse/SPARK-25444 Project: Spark

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611717#comment-16611717 ] Kazuaki Ishizaki commented on SPARK-20184: -- In {{branch-2.4}}, we still see the performance

[jira] [Commented] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611633#comment-16611633 ] Kazuaki Ishizaki commented on SPARK-16196: -- [~cloud_fan] This PR in the Jira entry proposes two

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611502#comment-16611502 ] Kazuaki Ishizaki commented on SPARK-20184: -- Although I created another JIRA https

[jira] [Commented] (SPARK-16196) Optimize in-memory scan performance using ColumnarBatches

[ https://issues.apache.org/jira/browse/SPARK-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611494#comment-16611494 ] Kazuaki Ishizaki commented on SPARK-16196: -- I see. I will check this. > Optimize in-mem

[jira] [Created] (SPARK-25388) checkEvaluation may miss incorrect nullable of DataType in the result

Kazuaki Ishizaki created SPARK-25388: Summary: checkEvaluation may miss incorrect nullable of DataType in the result Key: SPARK-25388 URL: https://issues.apache.org/jira/browse/SPARK-25388

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604120#comment-16604120 ] Kazuaki Ishizaki commented on SPARK-25317: -- When I have been investigating this issue, I

[jira] [Created] (SPARK-25338) Several tests miss calling super.afterAll() in their afterAll() method

Kazuaki Ishizaki created SPARK-25338: Summary: Several tests miss calling super.afterAll() in their afterAll() method Key: SPARK-25338 URL: https://issues.apache.org/jira/browse/SPARK-25338

Re: Spark JIRA tags clarification and management

Of course, we would like to eliminate all of the following tags "flanky" or "flankytest" Kazuaki Ishizaki From: Hyukjin Kwon To: dev Cc: Xiao Li , Wenchen Fan Date: 2018/09/04 14:20 Subject:Re: Spark JIRA tags clarification and management Thank

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602567#comment-16602567 ] Kazuaki Ishizaki commented on SPARK-25317: -- I confirmed this performance difference even after

[jira] [Commented] (SPARK-25317) MemoryBlock performance regression

[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602506#comment-16602506 ] Kazuaki Ishizaki commented on SPARK-25317: -- Let me run this on 2.3 and master. One question

[jira] [Updated] (SPARK-25317) MemoryBlock performance regression

[ https://issues.apache.org/jira/browse/SPARK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25317: - Description: eThere is a performance regression when calculating hash code

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Description: Invoking {{ArraysOverlap}} function with non-nullable array type throws

[jira] [Updated] (SPARK-25310) ArraysOverlap may throw a CompileException

[ https://issues.apache.org/jira/browse/SPARK-25310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25310: - Summary: ArraysOverlap may throw a CompileException (was: ArraysOverlap throws

[jira] [Created] (SPARK-25310) ArraysOverlap throws an Exception

Kazuaki Ishizaki created SPARK-25310: Summary: ArraysOverlap throws an Exception Key: SPARK-25310 URL: https://issues.apache.org/jira/browse/SPARK-25310 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25178) Directly ship the StructType objects of the keySchema / valueSchema for xxxHashMapGenerator

[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25178: - Summary: Directly ship the StructType objects of the keySchema / valueSchema

[jira] [Commented] (SPARK-25178) Use dummy name for xxxHashMapGenerator key/value schema field

[ https://issues.apache.org/jira/browse/SPARK-25178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587897#comment-16587897 ] Kazuaki Ishizaki commented on SPARK-25178: -- [~rednaxelafx] Thank you for opening a JIRA entry

[jira] [Updated] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25036: - Description: When compiling with sbt, the following errors occur: There are -two

[jira] [Comment Edited] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575137#comment-16575137 ] Kazuaki Ishizaki edited comment on SPARK-25036 at 8/9/18 5:05 PM

[jira] [Reopened] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

[ https://issues.apache.org/jira/browse/SPARK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-25036: -- Another type of compilation error is found > Scala 2.12 issues: Compilation er

[jira] [Commented] (SPARK-25059) Exception while executing an action on DataFrame that read Json

[ https://issues.apache.org/jira/browse/SPARK-25059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575129#comment-16575129 ] Kazuaki Ishizaki commented on SPARK-25059: -- Thank you for reporting the issue. Could you please

[jira] [Updated] (SPARK-25041) genjavadoc-plugin_0.10 is not found with sbt in scala-2.12

[ https://issues.apache.org/jira/browse/SPARK-25041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-25041: - Summary: genjavadoc-plugin_0.10 is not found with sbt in scala-2.12 (was: genjavadoc

[jira] [Created] (SPARK-25041) genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12

Kazuaki Ishizaki created SPARK-25041: Summary: genjavadoc-plugin_2.12.6 is not found with sbt in scala-2.12 Key: SPARK-25041 URL: https://issues.apache.org/jira/browse/SPARK-25041 Project: Spark

[jira] [Created] (SPARK-25036) Scala 2.12 issues: Compilation error with sbt

Kazuaki Ishizaki created SPARK-25036: Summary: Scala 2.12 issues: Compilation error with sbt Key: SPARK-25036 URL: https://issues.apache.org/jira/browse/SPARK-25036 Project: Spark Issue

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570440#comment-16570440 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen][~skonto] Thank you for your

[jira] [Commented] (SPARK-25029) Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors

[ https://issues.apache.org/jira/browse/SPARK-25029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569621#comment-16569621 ] Kazuaki Ishizaki commented on SPARK-25029: -- [~srowen] I see. The following parts have

[jira] [Created] (SPARK-24962) refactor CodeGenerator.createUnsafeArray

Kazuaki Ishizaki created SPARK-24962: Summary: refactor CodeGenerator.createUnsafeArray Key: SPARK-24962 URL: https://issues.apache.org/jira/browse/SPARK-24962 Project: Spark Issue Type

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560600#comment-16560600 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~ericfchang] Thank you very much for your

[jira] [Created] (SPARK-24956) Upgrade maven from 3.3.9 to 3.5.4

Kazuaki Ishizaki created SPARK-24956: Summary: Upgrade maven from 3.3.9 to 3.5.4 Key: SPARK-24956 URL: https://issues.apache.org/jira/browse/SPARK-24956 Project: Spark Issue Type

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559987#comment-16559987 ] Kazuaki Ishizaki commented on SPARK-24895: -- I see. Thank you very much. At first, I will try

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559974#comment-16559974 ] Kazuaki Ishizaki commented on SPARK-24895: -- [~yhuai] Thank you. BTW, how can I re-enable

[jira] [Commented] (SPARK-24925) input bytesRead metrics fluctuate from time to time

[ https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559972#comment-16559972 ] Kazuaki Ishizaki commented on SPARK-24925: -- Do we need a test case or which test case covers

[jira] [Commented] (SPARK-24841) Memory leak in converting spark dataframe to pandas dataframe

[ https://issues.apache.org/jira/browse/SPARK-24841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552231#comment-16552231 ] Kazuaki Ishizaki commented on SPARK-24841: -- Thank you for reporting an issue with heap

[jira] [Commented] (SPARK-24754) Minhash integer overflow

[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535657#comment-16535657 ] Kazuaki Ishizaki commented on SPARK-24754: -- In test cases, we would appreciate it if you

Re: [SPARK ML] Minhash integer overflow

value generated by Spark with it generated by other implementations? Regards, Kazuaki Ishizaki From: Sean Owen To: jiayuanm Cc: dev@spark.apache.org Date: 2018/07/07 15:46 Subject:Re: [SPARK ML] Minhash integer overflow I think it probably still does its.job; the hash

Re: [SPARK ML] Minhash integer overflow

Thank for you reporting this issue. I think this is a bug regarding integer overflow. IMHO, it would be good to compute hashes with Long. Would it be possible to create a JIRA entry? Do you want to submit a pull request, too? Regards, Kazuaki Ishizaki From: jiayuanm To: dev

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530640#comment-16530640 ] Kazuaki Ishizaki commented on SPARK-24579: -- I cannot see comments on the doc, too. > S

Re: [Help] Codegen Stage grows beyond 64 KB

83647". The log has to include the all of the generated Java methods. The community may take more time to address this problem than the case with the small program. Best Regards, Kazuaki Ishizaki From: Aakash Basu To: Kazuaki Ishizaki Cc: vaquar khan , Eyal Zituny , user Date:

Re: [Help] Codegen Stage grows beyond 64 KB

that the community will address this problem. Best regards, Kazuaki Ishizaki From: vaquar khan To: Eyal Zituny Cc: Aakash Basu , user Date: 2018/06/18 01:57 Subject:Re: [Help] Codegen Stage grows beyond 64 KB Totally agreed with Eyal . The problem is that when Java

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517097#comment-16517097 ] Kazuaki Ishizaki commented on SPARK-24498: -- [~maropu] thank you, let us use this as a start

[jira] [Commented] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509569#comment-16509569 ] Kazuaki Ishizaki commented on SPARK-24529: -- I am working for this > Add spotbugs into ma

[jira] [Created] (SPARK-24529) Add spotbugs into maven build process

Kazuaki Ishizaki created SPARK-24529: Summary: Add spotbugs into maven build process Key: SPARK-24529 URL: https://issues.apache.org/jira/browse/SPARK-24529 Project: Spark Issue Type

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506239#comment-16506239 ] Kazuaki Ishizaki commented on SPARK-24498: -- Hi [~smilegator] Definetely, I am interested

[jira] [Commented] (SPARK-24486) Slow performance reading ArrayType columns

[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504845#comment-16504845 ] Kazuaki Ishizaki commented on SPARK-24486: -- Thank you for reporting a problem. Could you please

Re: Strange codegen error for SortMergeJoin in Spark 2.2.1

Thank you for reporting a problem. Would it be possible to create a JIRA entry with a small program that can reproduce this problem? Best Regards, Kazuaki Ishizaki From: Rico Bergmann To: "user@spark.apache.org" Date: 2018/06/05 19:58 Subject:Strange cod

[jira] [Created] (SPARK-24452) long = int*int or long = int+int may cause overflow.

Kazuaki Ishizaki created SPARK-24452: Summary: long = int*int or long = int+int may cause overflow. Key: SPARK-24452 URL: https://issues.apache.org/jira/browse/SPARK-24452 Project: Spark

[jira] [Created] (SPARK-24323) Java lint errors

Kazuaki Ishizaki created SPARK-24323: Summary: Java lint errors Key: SPARK-24323 URL: https://issues.apache.org/jira/browse/SPARK-24323 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480482#comment-16480482 ] Kazuaki Ishizaki commented on SPARK-24314: -- I am working for this. > interpreted element

[jira] [Reopened] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-24314: -- > interpreted element_at or GetMapValue does not work for complex ty

[jira] [Updated] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-24314: - Summary: interpreted element_at or GetMapValue does not work for complex types

[jira] [Resolved] (SPARK-24314) interpreted array_position does not work for complex types

[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24314. -- Resolution: Duplicate > interpreted array_position does not work for complex ty

[jira] [Created] (SPARK-24314) interpreted array_position does not work for complex types

Kazuaki Ishizaki created SPARK-24314: Summary: interpreted array_position does not work for complex types Key: SPARK-24314 URL: https://issues.apache.org/jira/browse/SPARK-24314 Project: Spark

[jira] [Commented] (SPARK-24273) Failure while using .checkpoint method

[ https://issues.apache.org/jira/browse/SPARK-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475941#comment-16475941 ] Kazuaki Ishizaki commented on SPARK-24273: -- Thank you for reporting this issue? Would

[jira] [Commented] (SPARK-24220) java.lang.NullPointerException at org.apache.spark.sql.execution.UnsafeExternalRowSorter.(UnsafeExternalRowSorter.java:83)

[ https://issues.apache.org/jira/browse/SPARK-24220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472195#comment-16472195 ] Kazuaki Ishizaki commented on SPARK-24220: -- Thank you for reporting an issue. Would

Re: SparkR test failures in PR builder

are appreciated. Regards, Kazuaki Ishizaki From: Joseph Bradley <jos...@databricks.com> To: dev <dev@spark.apache.org> Cc: Hossein Falaki <hoss...@databricks.com> Date: 2018/05/03 07:31 Subject:SparkR test failures in PR builder Hi all, Does anyo

[jira] [Commented] (SPARK-24119) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458297#comment-16458297 ] Kazuaki Ishizaki commented on SPARK-24119: -- It seems to make sense. It would be good to set

[jira] [Comment Edited] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452404#comment-16452404 ] Kazuaki Ishizaki edited comment on SPARK-23933 at 4/25/18 6:48 PM

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452404#comment-16452404 ] Kazuaki Ishizaki commented on SPARK-23933: -- Thank you for your comment. The current map can take

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450262#comment-16450262 ] Kazuaki Ishizaki commented on SPARK-23933: -- cc [~smilegator] > High-order function: map(ar

[jira] [Commented] (SPARK-10399) Off Heap Memory Access for non-JVM libraries (C++)

[ https://issues.apache.org/jira/browse/SPARK-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446166#comment-16446166 ] Kazuaki Ishizaki commented on SPARK-10399: -- https://issues.apache.org/jira/browse/SPARK-23879

[jira] [Comment Edited] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436976#comment-16436976 ] Kazuaki Ishizaki edited comment on SPARK-23933 at 4/18/18 4:22 PM

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441268#comment-16441268 ] Kazuaki Ishizaki commented on SPARK-23933: -- ping [~smilegator] > High-order function: map(ar

[jira] [Commented] (SPARK-23986) CompileException when using too many avg aggregation after joining

[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438885#comment-16438885 ] Kazuaki Ishizaki commented on SPARK-23986: -- While I also checked it with branch-2.3, it works

[jira] [Comment Edited] (SPARK-23986) CompileException when using too many avg aggregation after joining

[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438818#comment-16438818 ] Kazuaki Ishizaki edited comment on SPARK-23986 at 4/15/18 7:36 PM

[jira] [Commented] (SPARK-23986) CompileException when using too many avg aggregation after joining

[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438818#comment-16438818 ] Kazuaki Ishizaki commented on SPARK-23986: -- Thank for reporting an issue with deep dive. When I

[jira] [Created] (SPARK-23976) UTF8String.concat() or ByteArray.concat() may allocate shorter structure.

Kazuaki Ishizaki created SPARK-23976: Summary: UTF8String.concat() or ByteArray.concat() may allocate shorter structure. Key: SPARK-23976 URL: https://issues.apache.org/jira/browse/SPARK-23976

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436976#comment-16436976 ] Kazuaki Ishizaki commented on SPARK-23933: -- [~smilegator] [~ueshin] Could you favor us? SparkSQL

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436136#comment-16436136 ] Kazuaki Ishizaki commented on SPARK-23933: -- I will work for this, thank you. > High-or

[jira] [Commented] (SPARK-23915) High-order function: array_except(x, y) → array

[ https://issues.apache.org/jira/browse/SPARK-23915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435666#comment-16435666 ] Kazuaki Ishizaki commented on SPARK-23915: -- I will work for this, thanks. > High-order funct

[jira] [Commented] (SPARK-23914) High-order function: array_union(x, y) → array

[ https://issues.apache.org/jira/browse/SPARK-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434494#comment-16434494 ] Kazuaki Ishizaki commented on SPARK-23914: -- I will work for this, thank you. > High-or

[jira] [Commented] (SPARK-23913) High-order function: array_intersect(x, y) → array

[ https://issues.apache.org/jira/browse/SPARK-23913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434491#comment-16434491 ] Kazuaki Ishizaki commented on SPARK-23913: -- I will work for this, thank you. > High-or

[jira] [Commented] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431795#comment-16431795 ] Kazuaki Ishizaki commented on SPARK-23916: -- Sorry for my mistake regarding a PR with wrong JIRA

[jira] [Comment Edited] (SPARK-23923) High-order function: cardinality(x) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431009#comment-16431009 ] Kazuaki Ishizaki edited comment on SPARK-23923 at 4/9/18 6:36 PM: -- I

[jira] [Commented] (SPARK-23921) High-order function: array_sort(x) → array

[ https://issues.apache.org/jira/browse/SPARK-23921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431037#comment-16431037 ] Kazuaki Ishizaki commented on SPARK-23921: -- I am working for this > High-order funct

[jira] [Comment Edited] (SPARK-23919) High-order function: array_position(x, element) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431004#comment-16431004 ] Kazuaki Ishizaki edited comment on SPARK-23919 at 4/9/18 6:19 PM: -- I

[jira] [Commented] (SPARK-23923) High-order function: cardinality(x) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431009#comment-16431009 ] Kazuaki Ishizaki commented on SPARK-23923: -- I am working for this. > High-order funct

[jira] [Commented] (SPARK-23919) High-order function: array_position(x, element) → bigint

[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431004#comment-16431004 ] Kazuaki Ishizaki commented on SPARK-23919: -- I am working for this. > High-order funct

[jira] [Commented] (SPARK-23924) High-order function: element_at

[ https://issues.apache.org/jira/browse/SPARK-23924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431002#comment-16431002 ] Kazuaki Ishizaki commented on SPARK-23924: -- I will work for this. > High-order funct

[jira] [Created] (SPARK-23893) Possible overflow in long = int * int

Kazuaki Ishizaki created SPARK-23893: Summary: Possible overflow in long = int * int Key: SPARK-23893 URL: https://issues.apache.org/jira/browse/SPARK-23893 Project: Spark Issue Type

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Description: The following code in {{UTF8StringSuite}} has no sense. {code

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Description: The following code in {{UTF8StringSuite}} has no sense. {code

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Summary: Improve coverage and fix lint error in UTF8String-related Suite (was: Improve

[jira] [Created] (SPARK-23892) Improve coverage and fix lint error in UTF8StringSuite

Kazuaki Ishizaki created SPARK-23892: Summary: Improve coverage and fix lint error in UTF8StringSuite Key: SPARK-23892 URL: https://issues.apache.org/jira/browse/SPARK-23892 Project: Spark

[jira] [Created] (SPARK-23882) Is UTF8StringSuite.writeToOutputStreamUnderflow() supported?

Kazuaki Ishizaki created SPARK-23882: Summary: Is UTF8StringSuite.writeToOutputStreamUnderflow() supported? Key: SPARK-23882 URL: https://issues.apache.org/jira/browse/SPARK-23882 Project: Spark

[jira] [Updated] (SPARK-23762) UTF8StringBuilder uses MemoryBlock

[ https://issues.apache.org/jira/browse/SPARK-23762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23762: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23879 > UTF8StringBuil

<    1   2   3   4   5   6   7   8   9   10   >